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Abstract 



Evaluating an Organizational Protocol to Implement Effective Leadership Meetings. 
Anderson, Erika, 2009: Applied Dissertation, Nova Southeastern University, Fischler 
School of Education and Human Services. Protocol Materials/Teacher Collaboration/ 
Leadership Training/Collaborative Teachers 

The problem on which this study focused was the need to organize and implement 
effective leadership meetings that led to improved professional collaboration. The 
purpose was to evaluate the use of a protocol in organizing and implementing such 
meetings. 

The research questions were framed around the four levels of the Kirkpatrick (1996) 
evaluation model: 

1. What are the reactions of the leadership team members to the protocol as a 
professional-development tool? 

2. What skills and knowledge related to organizing and implementing effective leadership 
meetings will the leadership team members learn as a result of using the protocol as a 
professional-development tool? 

3. Will leadership team members routinely and consistently apply the new skills and 
knowledge related to organizing and implementing effective leadership team meetings in 
their meetings? 

4. What result will learning the protocol have on participants’ judgments about 
organizing and implementing effective leadership meetings? 

An 1 1 -member leadership team participated in two professional-development training 
sessions and one simulation to improve their understanding of how to organize and 
implement effective leadership meetings. Prior to any staff development, the team was 
administered the Professional Learning Community Assessment (Huffman & Hipp, 

2003). The first training session addressed the basic look and sound of professional 
collaboration and was evaluated using the Professional Development Evaluation Survey 
(Steele, 2007). The second training involved the use and purpose of the protocol. 
Reflective journals were kept and rubric analyses conducted throughout the study to 
monitor the effectiveness of the protocol during leadership meetings. 

Additional data were gathered at the conclusion of the study with the readministration of 
the Professional Learning Community Assessment (Huffman & Hipp, 2003). This was an 
effort to determine the impact of using the protocol to organize and implement effective 
leadership meetings. As anticipated, the leadership team experienced overall success in 
organizing and implementing an effective leadership meeting by using the specific 
protocol. 
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Chapter 1: Introduction 

Introduction 

This paper presents a description of a research study designed to organize and 
implement effective leadership meetings. The dissertation includes five chapters. Chapter 
1 provides an overview of this study. Included in this chapter is a discussion of the 
problem and purpose of this study. Chapter 1 also states the research questions and 
defines terms used in the study. Chapter 2 reviews literature and research documents that 
are related to the problem and the intervention for this study. Chapter 3 provides a 
description of the research design and methodology used. Chapter 4 presents the results 
for each research question. Chapter 5 discusses the results of the data obtained through 
this research study in detail, referring to current research findings to validate findings. 
Chapter 5 also provides recommendations for future research. 

This study was conducted at a rural middle school located on the outskirts of a 
rapidly growing city in the southeastern United States. The problem on which this study 
focused was the need to improve the organization and effectiveness of leadership 
meetings. The researcher implemented and evaluated professional development provided 
to the leadership team about the behaviors of professional collaboration. The leadership 
team was also instructed on the purpose, benefits, and use of the state’s suggested 
protocol called Creating the Capacity for Change Through Action Research (CCCTAR), 
which was developed by the state’s Regional Educational Service Agency (RES A; 2002). 
Protocols contain segments that facilitate the steps of talking and listening, describing 
and judging, and reaching a consensus in decisions. 

Most protocols are designed as prescribed steps that force transparency in 
conversations, which means that all parties understand the message or thoughts being 
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shared by others. The messages are made clear to the listener because opportunities are 
presented to question the speaker, hence allowing for clarity (i.e., transparency). They 
segment elements of the conversation where it otherwise may have blurred (McDonald, 
Mohr, Dichter, & McDonald, 2007). The leadership team practiced the CCCTAR 
protocol (RESA, 2002) through a simulation. This allowed the leadership team members 
the opportunity to use collaborative behaviors in a protected environment prior to any 
assessment of such behaviors. 

The researcher’s state promoted the use of professional development as a method 
that was fundamental to school improvement (Georgia Department of Education, 2007c). 
In an effort to be rated as operational, the target school’s leadership team needed to 
improve their professional collaboration to achieve common goals and promote 
organizational effectiveness (Georgia Department of Education, 2007a). The study was a 
mixed-methods study using the Kirkpatrick (1996) four levels of evaluation model as a 
framework for evaluation. 

Description of the Setting 

The target school was located in an area with a diverse population. The quality of 
housing for residents varied; some families lived in upper middle class neighborhoods, 
and others resided in single-family homes with annual incomes that averaged $42,000 or 
less a year. The target school’s lead counselor shared that there was a growing population 
of migrant workers who resided in trailer parks and multiple families living in single- 
family dwellings (T. Vincent, personal communication, March 1, 2008). 

The facility originally opened in 1957 as a high school. In 1972, it was converted 
to a junior high school. The building was then renovated and expanded in 1993, allowing 
for the addition of the sixth grade, which would serve as a middle school. The school 
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served 920 students and offered a traditional middle school program. 

The student-to-teacher ratio was 16:1 at the target school. The state average was 
13:1. Because the families of some students worked for poultry plants, construction 
companies, and seasonal landscaping businesses, students of the target school missed 
more school on average than students of other schools, as measured by the state. At the 
target school, an average of 53% of students missed five or fewer days, compared to the 
state average of 61% of students missing five or fewer days per school year (Great 
Schools, 2006). Families often relocated where work could be found, which meant 
pulling students from the classroom setting. Housing was temporary for many of the 
seasonal workers. For an immeasurable number of varying reasons, students’ families 
were either evicted or displaced during the school year. Too often, these families did not 
withdrawal their children from the target school; instead, they were left on the class rolls 
and appeared as chronic absentee problems. 

Students who were eligible for the free and reduced-price lunch program 
exceeded 69% of the student body. This figure reflected the socioeconomic status of the 
majority of the school’s student population. Statistically, 48% of the population was 
Hispanic. The second largest ethnic group included White, non-Hispanic students, 
statistically represented by 39%. According to Great Schools (2006), the rest of the 
student body was as follows: Black (8%), multiracial (2%), and Asian (3%). 

The faculty of the target school was predominantly White. Of the 86 certified 
staff, five members were African American, and two members were Hispanic. Thirty-five 
percent of the certified staff held bachelor’s degrees, 52% held master’s degrees, and 
12% held specialist degrees. One faculty member held a doctoral degree (Great Schools, 



2006). 
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The researcher served as a member of the faculty at the target school. Having 
served on the leadership team for 2 consecutive years, the researcher had witnessed 
firsthand the lack of professional collaboration. Serving as a curriculum chairperson from 
2004 to 2009, the researcher observed how professional collaboration had not been 
achieved at the target school. The weekly meetings on the Georgia Performance 
Standards (GPS) often ran overtime and did not meet the objectives set in the agenda. 
When discussing goals for students, members frequently varied in their opinions of what 
was expected. Participants became defensive when their opinions were not shared with 
the group. Comments were often taken as personal attacks. Feelings had been hurt, and 
animosity began to develop among the participants. 

Statement of the Problem 

As stated above, the problem on which this study focused was the need to 
organize and implement effective leadership meetings, which were designed to assist 
school personnel in making needed changes in curriculum instruction to improve student 
achievement. The discussion revolved around new strategies, technology, recent studies, 
and other items that directly affected the students at the target school. Although there was 
no set agenda designated by the state for these meetings, the overall goal was to improve 
student achievement as measured by the state. 

The target school did not meet annual yearly progress (AYP) for the 2007-2008 
school year and had been placed in mandatory reform (Georgia Department of Education, 
2007a). Having been named as a school that needed improvement, the target school 
struggled to meet AYP and reestablish a reputable standing with the state department of 
education. When a public school failed to meet AYP, the state department of education 
asserted that with the accountability requirements of the No Child Left Behind Act of 
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2001, schools had to evaluate and monitor their programs through data-driven, research- 
based practices. The purpose of the Georgia Assessment of Performance School 
Standards (GAPSS) analysis was to provide a process of data collection and verification 
of a school’s status and offer specific direction for school improvement in a variety of 
areas (Georgia Department of Education, 2007b). Included in this directive was the 
monitoring of the school leadership team. 

In Fall 2007, the GAPSS review committee rated the leadership of the target 
school, which included the leadership team, as performing at an emergent level. This 
meant that, overall, little evidence existed to confirm the efforts of administration or 
faculty in meeting criteria in the area of leadership. This was decided by the review 
committee after a 3-day visit to the target school. The GAPSS review committee also 
interviewed leadership team members, reviewed past agenda, and looked for evidence 
that goals were met. The evidence appeared in the form of (a) paper documents, (b) new 
procedures used at the school, and (c) personnel reflections from members. Significant 
areas that were deemed emergent by the GAPSS review committee included problem 
solving, decision making, distributed leadership, experiences created, and team approach. 

The impact of this problem was that the target school was unable to earn even an 
operational ranking from the GAPSS review committee in the area of school leadership. 
The evaluations rankings were (a) fully operational (the highest level of achievement), 

(b) operational, (d) emergent, and (e) not yet (Georgia Department of Education, 2007a). 
There were no areas ranked not yet, which was the lowest possible rank. 

Criteria for each of the seven areas of leadership evaluated by the state 
department of education are provided in the next section of this paper. The goal of the 
target school was to be operational in the area of school leadership, meaning that the 
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administration and faculty were making significant measurable efforts to either obtain a 
particular skill or meet stated criteria. Although the intent of this study was to help the 
leadership team move toward this goal, the next GAPSS review, which would determine 
whether the goal had been met, would occur beyond the time frame of this study. 

The target school’s administration was expected to implement the suggestions of 
the GAPSS review committee in the area of school leadership. Evidence of such efforts 
would be submitted to the state department of education twice a year; this was a 3-year 
process. This study provided a portion of the documented evidence needed for improving 
the school leadership’s team efforts to improve collaboration and shared decision-making 
processes. Specifically, the intent of this study was to implement and evaluate the use of 
the CCCTAR protocol (RESA, 2002) intended to organize and implement effective 
leadership meetings. 

It is important to note that the use of the protocol used in this study had a 
measurable impact on the team’s ability to collaborate professionally and effectively. 
That was the fundamental assumption on which this applied dissertation study was based. 
Effective communication is essential to professional collaboration. Protocols force 
transparency, implying openness in communication, and create accountability in such a 
way that biases and misunderstandings can be addressed in nonconfrontational 
environments (McDonald et al., 2007). In addition, protocols accelerate the development 
of facilitative leadership and help to develop the attitudes, mindset, and skills of teachers 
as they work collaboratively to attain preset goals (McDonald et al., 2007). 

Background and Significance of the Problem 

As stated in the previous section, the target school did not meet AYP for the 
2007-2008 school year; therefore, the school was in mandatory reform for the 2008-2009 
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school year (Georgia Department of Education, 2007a). Along with state funds, the target 
school had received directives to improve student learning. One of the directives related 
to the competence of an effective school leadership team, as described by the Georgia 
Department of Education (2007a): 

A school leadership team is operational and consists of representation of the 
school staff. The team needs to expand its emphasis upon collaborative decision 
making and problem solving to enhance staff and student achievement, thereby 
resulting in decreased problems associated with organizational performance, (p. 
132) 

According to the state department of education’s Keys to Quality, the school 
leadership team has specific functions, one being the ability to determine appropriate 
programs and process as well as to analyze causes of academic and organizational 
problems and related solutions (Georgia Department of Education, 2007a). Except for a 
large binder that contains rules and regulations, and the means by which they will be 
measured, leadership teams are not given training as to how to achieve their goals. Once 
a school’s team is deemed as not having met a particular area with success, the state 
panel provides an explanation as to why a particular area was not met, but offers no 
concrete steps for future success. The leadership team scored overall in the emergent 
level after several areas were evaluated under leadership descriptors. The leadership areas 
as defined by the Georgia Department of Education (2007 d) can be found in Appendix A. 

The leadership team did meet success in some areas, such as governance, where 
they received operational status. Because of the roles of GPS lead teachers, the school 
leadership team also received operational marks in instructional leadership development. 
These GPS roles were filled by faculty members who hosted weekly meetings to develop 
units of study that meet the standards developed by the state department of education. 
Although the position of GPS lead teacher was a volunteer position with no incentives, 
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each school was required to have a person serve in this position in each of the curriculum 
areas at each grade level. 

Teacher leaders chosen or volunteering to lead did not have any specific training. 
Upon initial implementation of this role 6 years ago, chosen leaders attended a workshop. 
In the workshop, which was hosted by the county school system, background knowledge 
on the state standards induction was addressed. Manuals were given that provided 
scaffolding on understanding how to teach to the standards. Although the county school 
system provided the opportunity for teacher leaders to learn how to teach students using 
new techniques, there was no training on how to teach other teachers. The thought of the 
administration was that once the teacher leaders understood the standard way of 
instruction, they could then model and explain it to other teacher in their curriculum 
areas. 

Fullan (1993) emphasized the importance of professional collaboration by stating, 

“Without collaborative skills and relationships, it is not possible to learn and to continue 

to learn as much as you need in order to be an agent of social improvement” (pp. 17-18). 

Steele (2007) pointed out the following: 

Collaborative dialogue has been defined by Burton, Burna, and Treasure-Jones 
(1997) as collaboration that is maintained by a set of implicit obligations or 
beliefs, such that each participant believes that the other participants can make 
significant contributions to the solution of the current task, or that it is polite to 
answer in an informative manner, (p. 9) 

According to the Southwest Educational Development Laboratories (2008), “In 
order to help low-performing schools become communities of continuous inquiry and 
improvement, one must first acknowledge and understand the issues that are affecting the 
school’s efforts to make improvements” (Introduction section, ^l). The ability of the 
target school’s leadership team to professionally collaborate was an area in which 
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improvements needed to be made. 

The GAPSS review committee (Georgia Department of Education, 2008) stated, 
“The school leadership team tends to be centrally controlled and directed, with minimal 
evidence of collaborative decision making and problem solving” (p. 85). Further, the 
GAPSS review committee documented that “the team needs to expand its emphasis upon 
collaborative decision making and problem solving to enhance staff and student 
achievement, therefore resulting in decreased problems associated with organizational 
performance” (Georgia Department of Education, 2008, p. 85). 

A possible cause for the leadership team at the target school to be struggling in 
professional collaboration may have been how the leadership team was once comprised. 
The leadership team consisted of three members representing the different curriculum 
areas (e.g., Language Arts, Math, Science, and Social Studies), one member representing 
special interest subjects (e.g., Consumer Science, Band, Chorus, Foreign Language, and 
Computer), one member representing the Physical Education Department, one member 
from the Special Education Department, one member from the English Speakers of Other 
Languages Department, the school’s Instructional Lead Teacher, two assistant principals, 
and the school principal. Between 2000 and 2007, the members of the leadership team 
were random participants who had volunteered. Leadership team participation was 
viewed by the faculty as a duty, but the team had no real responsibility. Opinions of the 
leadership team members were not sought by administration, and suggestions of the 
teachers serving on the leadership team were not implemented. The principal stated that 
she hosted each meeting and did a majority of the talking (P. Stubbs, personal 
communication, April 15, 2008). 

During the 2006-2007 school year, the principal placed an importance on the 




10 



quality of the participants in the leadership team and asked that each department vote and 
select a member to represent them. There had been a shift in who was on the leadership 
team. The responsibilities and obligations, and how meetings were conducted, also 
changed in the 2007-2008 school year. The principal reported that she still set an agenda, 
but the participants were involved in discussions and decisions that only had a direct 
effect on curriculum and management of the school (P. Stubbs, personal communication, 
April 15, 2008). 

Although efforts had been made to develop quality leaders, the school still failed. 
Through the GAPSS evaluation process, a school leadership team was not considered 
fully operational until “the school leadership team had developed and consistently used a 
protocol for handling business, making decisions, and solving problems effectively and 
collaboratively related to all facets of student needs, staff productivity, and organizational 
performance” (Georgia Department of Education, 2008, p. 85). The principal had asked 
that a protocol be implemented and utilized (P. Stubbs, personal communication, January 
15, 2008). She had asked for this upon the recommendations of the state department of 
education. However, there was no specific protocol assigned for use by the state. 

During this study, the researcher did not serve in any leadership position. She 
continued to instruct the language arts, but did not head the department. Her position on 
the leadership team was filled by a faculty member wishing to serve in this capacity. The 
researcher’s influence over this study was limited to observing and evaluating the 
implementation and effectiveness of the CCCTAR protocol (RESA, 2002) because it was 
used in an effort to organize and implement effective leadership meetings. However, the 
researcher assisted in the implementation of professional development with the leadership 
team by coaching the instructional lead teacher on the protocol’s purpose and procedures. 
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One of the responsibilities of the lead teacher at the target school was implementing and 
hosting staff developments. He had been trained in this area and was certified in 
educational leadership. Using research from this study, the lead teacher and the 
researcher created the needed materials to implement the staff development. The 
instructional lead teacher had also agreed to host professional development on what 
professional collaboration looks and sounds like. The researcher observed the meetings 
and took anecdotal notes. Hosting the professional-development sessions would not have 
allowed the researcher to make any documented evaluation during the training sessions. 
Purpose of the Study 

The purpose of this study was to organize and implement effective leadership 
meetings. The CCCTAR protocol (RES A, 2002) was used to carry out the objectives of 
this applied dissertation. The following research questions provided a framework from 
which this study followed. 

Research Questions 

The research questions listed below guided this study and were framed around the 
four levels of the Kirkpatrick (1996) model for evaluating the effectiveness of 
professional development: 

1. What are the reactions of the leadership team members to the protocol as a 
professional-development tool? This was measured using the Professional Development 
Evaluation Survey (PDES; Steele, 2007), which was completed after the two training 
sessions. Research Question 1 addressed reaction, which is Level 1 of the Kirkpatrick 
(1996) model. 

2. What skills and knowledge related to organizing and implementing effective 
leadership meetings will the leadership team members learn as a result of using the 
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protocol as a professional-development tool? This was measured using observations of 
the leadership team meetings, responses in reflective journals, and use of the rubric for 
Elements of an Effective Collaborative Group (Danielson, 2002). Research Question 2 
addressed learning, which is Level 2 of the Kirkpatrick (1996) model. 

3. Will leadership team members routinely and consistently apply the new skills 
and knowledge related to organizing and implementing effective leadership team 
meetings in their meetings? This was measured using observations of the leadership team 
meetings and responses shared in the reflective journals. Research Question 3 addresses 
behavior, which is Level 3 of the Kirkpatrick (1996) model. 

4. What result will learning the protocol have on participants’ judgments about 
organizing and implementing effective leadership meetings? This was measured using 
reflective journals and pretest and posttest surveys that addressed members’ perceptions 
of roles and professional-collaboration abilities experienced during meetings. Research 
Question 4 addressed results, which is Level 4 of the Kirkpatrick (1996) model. 
Definition of Terms 

Lor the purpose of this applied dissertation, the following terms are defined. 

Collaborating. This term refers to the use of dialogue. According to Chawla and 
Renesch (1995), “dialogue is most useful for learning about complexity where no one has 
the answer (p. 176). Dialogue is the sharing of knowledge or thoughts in an effort to seek 
a solution to a problem. 

Conversing. This term refers to talking or having a conversation in which 
information is broken into parts by the participants. Those conversing share what they 
know or feel about a particular subject (Chawla & Renesch, 1995). 

Dependent variable. Lor this study, the dependent variable was the knowledge 
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and application of skills learned by the leadership team, which was demonstrated by 
organized and effective leadership meetings. 

Independent variable. For this study, the independent variable was the 
professional development that was delivered to the leadership team. 

Professional collaboration. This term refers to “working as a team and valuing 
different perspectives” (Casey House, 2008, p. 1). 

Protocols. This term refers to steps in talking and listening, describing and 
judging, and reaching a consensus in decisions. Protocols are prescribed steps that force 
transparency in conversations. They segment elements of the conversation where they 
otherwise may have been blurred (McDonald et al., 2007). 

Summary 

Chapter 1 reflects the purpose of this applied dissertation. The study was intended 
to evaluate the use of a protocol on organizing and implementing an effective leadership 
meeting. This was needed for the leadership team to move toward improved 
collaboration, which was required for the members of the school leadership team to be 
deemed operational in their efforts, according the GAPSS evaluation team. 

This chapter addressed the significant issues faced by the leadership team in their 
efforts to become operational and why they had not yet achieved this ranking by the state 
department of education. The mandates from the state regarding the responsibilities of a 
school leadership team were given, as well as the steps that a team must take to become 
operational. The research questions that guided this study were included, as well as 
definitions that were significant to the understanding of contextual vocabulary. 
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Chapter 2: Review of the Literature 

Introduction 

Federal legislation, such as the NCLB (2001), has changed the way that public 
schools address competency reform efforts in education and has placed an importance on 
a school leadership team’s ability to achieve maximum results for the students for whom 
they have responsibility. Professional collaboration represented an area that federal 
legislators have cited as one promoting successful reformation. The federal rules and 
regulations of the NCLB, as well as state department of education directives, position 
papers, and professional-development funding, reflect the concept that professional 
collaboration is a critical factor in achieving successful educational reforms (Leonard & 
Leonard, 2003). 

This chapter discusses the effects of professional collaboration in school 
leadership teams, how to collaborate (including reflective practices) in school leadership 
teams, the relationship of collaboration to student learning, and causes of lack of 
collaboration among education professionals. In addition, the role of professional 
development in building effective collaboration and the Kirkpatrick (1996) training 
evaluation model are presented in chapter 2. Because the Kirkpatrick model guided the 
study, a review of the process is found in this chapter. The chapter concludes with a 
summary. 

Nature of the Problem 

Research by Hallinger and Heck (1996), Leithwood and Riehl (2003), Spillane, 
Halverson, and Diamond (2003), Cotton (2003), Elmore (2000), and Donaldson (2001) 
supported the idea that leadership was necessary to improve schools so that all students 
could benefit. In the past, bureaucratic hierarchical structures in education were focused 
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on the task of supporting efficiency and order within a school (Cotton, 2003; Donaldson, 
2001; Elmore, 2000; Hallinger & Heck, 1996; Leithwood & Riehl, 2003; Spillane et al., 
2003). There was the perception among the public that “teaching requires no expertise” 
(Elmore, 2000, p. 6). However, schools solely run by bureaucracies have developed a 
disconnection between what happens in classrooms and the administration (Bailey, 
Cameron, & Cortez- Ford, 2004). 

Schools need to adapt to societal demands for accountability and rapid change 
(Bailey et al., 2004). Shared decision making was thought to be a key factor in reforming 
curricula and transforming the work of teachers (Darling-Hammond, 1996). When 
schools are structured to facilitate collaboration and expand leadership roles, efficacy and 
the ability to meet students’ needs increase (Rosenholtz, 1998). Many schools have 
developed collaborative efforts through professional-learning communities, thereby 
allowing collaboration to occur; however, simply learning to collaborate is not sufficient 
to improve schools and learning. Bailey et al. (2004) found that collaboration needed to 
exceed the professional-learning community, and an emphasis should be placed on the 
leadership of the school. They found that collaboration lacking purpose, connection, and 
distributed leadership can have little effect on student leaning. 

According to Johnson (as cited in Lucas & Valentine, 2002), “principals are being 
asked to empower their teachers in order to take advantage of recent moves toward site- 
based management and shared decision-making structures (p. 2). Furthermore, Johnson 
reported, “The complexity of such requirements has led recent calls for reform to 
emphasize the need for collaborative leadership” (p. 3). Transformational leadership is 
about engaging and inspiring others to go beyond self-interest and work toward a value- 
driven and shared decision-making process (Tucker- Ladd, Merchant, & Thomas, 1992). 
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Lucas and Valentine (2002) studied 12 middle schools, seeking insight into 
transformational leadership, school leadership teams, school culture, and the relationship 
between them. The researchers found that leadership teams fostering a commitment to 
group goals, providing individualized support and intellectual stimulation, holding high 
expectations, identifying and articulating a vision, and providing an appropriate model 
for communication were successful. Leadership teams using a model with identified and 
articulated visions were also effective. However, collaboration without a purpose was 
meaningless. Lucas and Valentine demonstrated that the principal establishes what is 
expected from the faculty by his or her attitude toward the leadership team approach. 
With opportunity and welcome, collaboration allows for the opening of unforeseen doors, 
leading to positive change within a school. 

Importance of Teacher Collaboration 

Inger (1993) noted, “Most of the current major education reforms call for 
extensive, meaningful teacher collaboration” (p. 1). He also stated, “Teachers who have 
worked together see substantial improvements in student achievement, behavior, and 
attitude” (p. 5). Professional-learning communities “focus on learning rather than 
teaching, working collaboratively, and holding you accountable for results” (DuFour, 
2002, p. 13). Although a professional-learning community and collaboration are not 
interchangeable, collaboration comes about through a professional-learning community. 
Newmann and Wehlage (1995) stated, “If schools want to enhance their organizational 
capacity to boost student learning, they should work on building a professional 
community that is characterized by a shared purpose, collaborative activity and collective 
responsibility” (p. 37). 

Fullan (1993) stressed that the ability to collaborate was a core requirement of the 
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post-modern society. He reported further about the importance of collaborative skills and 
the ability to form and maintain relationships. Fullan thought that it was impossible to be 
an agent for social improvement without the ability to collaborate professionally. Others 
have contributed to this thought. DuFour and Eaker (1998) asserted that teachers could 
increase the effectiveness of the school when they collectively identified and worked 
toward a common goal. 

Reflective Practices 

The power and effectiveness of professional-learning communities comes from a 
group’s continuous inquiry and improvement (Southwest Educational Development 
Laboratories, 2008). The target school’s leadership team needed to improve their 
collaborative abilities. The RESA (2002) suggested evaluating the process through the 
use of reflective journals. Reflective practices are designed to increase an awareness of 
one’s professional performance (Blase & Blase, 2004). 

Reflective practices became an educational practice following the research of 
Schon (1983). The lecturing and conferences for teacher work days (i.e., professional- 
development days or units) gave way to the idea of encouraging professionals to talk and 
reflect upon situations and learn from their reflections and conversations. Working in 
collaborative groups allowed reflections to have relevance to the individual, and 
reflective practice became a cyclic learning experience. In reflecting, individuals are 
personally engaged in the process of learning (Appalachian Educational Laboratory, 
2005). 

Wallin (2003) found evidence linking a school’s effectiveness to the quality of the 
school’s leadership. He stated, “There have been literally thousands of research projects 
that attempt to identify the major correlates of the effective school” (p. 62). Wallin noted 
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that higher than expected results in academic subjects, as well in the social development 
of students, occurred in schools where teachers and principals were involved in reflective 
practices. The principal of a school is responsible for guiding a reflective process, thereby 
enabling significant and effective change. The principal initiates and demonstrates the 
need for and use of collaborative dialogue, reflective practice, and a systematic approach 
to professional teamwork. Although Wallin did not provide numerical data supporting 
these findings, he did develop an overview and examples of the behaviors successful in 
improving school effectiveness. 

Relationship of Teacher Collaboration to Student Learning 

In a research brief on shared leadership and student achievement (Appalachian 
Educational Laboratory, 2005), several studies were cited. Although not all of them 
found a significant positive correlation between shared leadership and student 
achievement, the studies supported the thought that slight benefits far outweigh any 
negative drawbacks to establishing shared leadership and collaborative efforts for school 
faculty to benefit student achievement. The brief presented by Appalachian Educational 
Laboratory stated the following: 

The performance expectations and accountability measures built into the No Child 
Left Behind Act are driving the need for a more systematic understanding of the 
ways that leadership may impact student achievement. Many studies have found 
an association between principal leadership behaviors and student academic 
performance, (p. 8) 

Marks and Printy (2003) emphasized the importance of teacher shared and 
instructional leadership in improving student performance. Tongeri and Anderson (2003) 
stated that a shared leadership approach would help a school navigate ambitious 
academic goals and facilitate student success. The Appalachian Educational Laboratory 
(2005) also published the following: 
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A 2003 survey of the distributed leadership literature conducted by the National 
College for School Leadership concluded: “The relationship between shared 
leadership and learning is a crucially important issue, but there are no empirical 
data at all on this” (Bennett, Wise, Woods, & Harvey, 2003, p. 12). The following 
year, however, Leithwood and colleagues (2004) published a review of the 
literature on how leadership influences student learning and concluded that there 
is an association between increased student learning and leaders who develop and 
rely on leadership contributions from a diverse constituent base within their 
development, (p. 6) 

Effective principals understand the role of a leader embracing the process as well 
as the results. DuFour (2002) concluded, “Only those who understand that the essence of 
their job is promoting student and teacher learning will be able to provide the leadership” 
(p. 18). An effective principal knows that collaboration is the core of a professional- 
learning community. Collaboration cannot be achieved without systemic intervention. 
Systemic intervention is a step-by-step guide to the what, how, when, and why of 
professional-learning activities that support participants in learning and applying new 
behaviors. A system of interventions for collaboration promotes learning, thereby 
allowing school faculty to focus not only on the effects of teacher collaboration, but also 
on the way that teachers interact effectively with their students. 

Lambert (1993) stated, “We know that unless teachers are learning together, they 
will not be able to create engaging learning experiences for children. Using that 
understanding will open a door and allow for many other ideas and skills to be 
implemented” (p. 93). In collaborative learning, Marquardt (2003) promoted the sharing 
of ideas and knowledge, thereby allowing learners the opportunity to review and learn 
from each other. Marquardt asserted that a collaborative group learns by utilizing an 
individual’s abilities as well as the team’s overall synergy. Smylie and Hart (1999) found 
positive relationships between teacher participation in professional collaboration and an 
increase in student responsibility and enthusiasm for learning and problem solving. 
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Although the amount of literature on the relationship of teacher collaboration to 
student learning had increased since 2005, articles and studies were still limited to 
leadership teams and their impact on student learning. Even after 83 studies on school- 
based management and student learning, there are no rigorous, scientifically based 
studies on the effects of school leadership on student learning (Appalachian Educational 
Laboratory, 2005). Although the studies may not consider the many facets that affect 
student learning, the studies were published results in the field of education. 

Reasons for Lack of Teacher Collaboration 

Even though an emphasis has been placed on teacher collaboration, the success of 
collaborative efforts has been rare (Inger, 1993). According to Inger, “there are many 
barriers to teacher collaboration, and the barriers are of many kinds” (p. 10). 

Furthermore, Leonard and Leonard (2003) stated, “Inhibitors to such collegial 
professional interaction have been noted often in the literature, among them time 
constraints, fragmented visions, competitiveness, conflict avoidance, and lack of 
administrative support” (p. 7). 

Leonard and Leonard (2003) conducted a study on teacher collaboration. Findings 
indicated that schools headed by administrators who value collaboration were successful 
in using the collaborative process. Schools that only told of the need to collaborate and 
did not have the structured support of administration failed at collaborative efforts. A 
leader of learners will use a protocol to help participants learn how to actively participate 
in effective collaboration. McDonald et al. (2007) stated, “Protocols help imagine 
alternatives to ordinary habits of working together, learning, and leading” (p. 15). 
According to Leonard and Leonard, “school principals who continue to personify 
traditional leader traits in the currently emerging educational environment not only 
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minimize professional growth, but they may also optimize student mediocrity” (p. 43). 
For collaborative efforts to have validity, knowing why to collaborate is equally 
important as knowing how to collaborate. In initiating collaboration in a school, a 
framework or set of guidelines can be useful. Protocols serve as the framework for 
effective collaboration and further substantiate positive change in any school. 

Protocol for Collaboration 

According to Kohm (2002), “what we don’t know can hurt us” (p. 1), indicating 
how important it is to have honest and critical feedback to communicate effectively 
within a school faculty. Seeking a positive change in a school often lies in the hands of 
the faculty. The power of any protocol is the emphasis on reflective practice, genuine 
listening, critical thinking, and feedback (Lambert, 2003). 

At Sir Winston Churchill High School, educators used protocols to examine the 
philosophies of curriculum leaders about education. They found that the protocols 
allowed them to hear more voices. More listening occurred, and, as a result, a deeper 
understanding of the issues emerged (Lambert, 2003). A protocol helps to develop the 
attitudes, mindset, and skills of teachers as they work collaboratively to attain 
predetermined goals. McDonald et al. (2007) began creating and supporting the use of 
protocols in education in 1991. Their first basic idea behind the use of protocols was that 
educators needed to take charge of their own learning. The authors felt that protocols 
could encourage an environment of learning. Protocols force transparency in 
conversations, allowing the participants to gain a deeper understanding of their 
colleagues’ opinions and insights. According to McDonald et al., “one of the values of 
using protocols as a learning format, in our view, is that they can accelerate the 
development of facilitative leadership and thus assist in the creation of new workplaces 
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for educators” (p. 13). 

Protocol for Creating Capacity for Change Through Action Research 

The CCCTAR protocol, which was created by the RESA (2002), organizes the 
structure of a meeting. The use of the adapted protocol is intended to implement effective 
leadership meetings. At the beginning of each session, the steps of the protocol are 
reviewed. Prior to a meeting, an agenda is sent to all participants. This allows time for 
participants to prepare for the topic of discussion for the meeting. Briefly, the steps for 
this protocol are the following: 

1. Introduction (5 minutes). The agenda is reviewed and goals are set for the 
meeting. 

2. Teacher preparation (5-10 minutes). The presenter describes the context of the 
meeting and shares any sample work needed to aid in the understanding of meeting’s 
purpose. 

3. Discussion (30 minutes suggested, but can be adjusted to suit needs of the 
group). The group actively participates to attain the goals of the meeting. 

4. Clarifying questions (10 minutes minimum). Participants reflect on what has 
been shared. Questions that seek clarity or summarize main points of the meeting are 
presented. The facilitator is the one to answer the questions based on his or her 
interpretation of key points shared by the group. 

5. Debriefing (5 minutes). The group discusses any frustrations, 
misunderstandings, or positive reactions to the session. 

6. Distributing journals. Each journal has the topic of discussion for each month 
stapled to the journal cover. The team member may retrieve the journal with anonymity 
because they are placed on a centrally located table out of the sight of the researcher 
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(RESA, 2002). 

Once a leadership team is established and has addressed perceptions, the team can 
begin the process of functioning with operational status, which is a desirable rating set by 
the state department of education. This is accomplished through the use of a prescribed 
process such as a protocol. In using a protocol, biases and misunderstandings are 
addressed in a nonconfrontational environment. Effective collaboration, leading to 
substantial change, is plausible. 

Professional Development 

Professional development “should be a collaborative endeavor with teachers and 
administrators working the plan and seeing its implementation through to the final 
stages” (Steele, 2007, p. 14). Meaningful professional development is centered on 
helping students achieve learning goals and supporting learning needs. The Georgia 
Department of Education (2007c) put into practice a professional-development plan with 
these same intentions. The plan was for all school administrators to focus teachers’ 
professional development around the goals and visions on the school’s improvement 
plan. The reasoning behind the plan was that the teachers’ sole purpose in developing 
professionally should be about meeting the needs of the students they instruct. A school 
improvement plan outlines the areas of need for each school and the appropriateness of 
focusing teachers’ professional development. 

The plan established by the Georgia Department of Education (2007c) gave 
directives to school administrations and provided support. Through the Professional 
Learning Services Unit, administrators and faculty were provided technical support, 
resources, and consulting services. With an emphasis placed on professional 
development, the state department of education created the following nine statements 
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(2007a): 

1. The primary purpose of professional learning is to improve learning of a 
diverse student population. 

2. Professional learning is fundamental to school’s improvement. 

3. Professional learning facilitates effective change and innovation in a mutually 
supportive environment. 

4. Professional learning is a shared process that promotes growth in individuals 
and organizations. 

5. Professional learning responds to the diverse needs of all personnel. 

6. Professional learning is an integral component of a school and a school system 
program. 

7. Planning and decisions in professional learning must include those who 
receive the training. 

8. Effective professional learning must be based on theory, research, and sound 
practice. 

9. Effective professional learning is responsive to the lifelong needs of adult 
learners. 

Pankake and Moller (as cited in Huffman & Hipp, 2003) discussed the 
educational dilemma facing President George W. Bush as he and state governors began to 
focus their attention on education reform. As Goals 2000 (NCLB, 2001) began to take 
shape, an emphasis was placed on the need for all school system leaders and employees 
to have goals for improvement. According to Pankake and Moller, eight measurable goals 
became the catalyst for improved professional development: (a) students ready to learn, 
(b) increased graduation rates, (c) expanding student competency in crucial areas, (d) 
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increased emphasis on math and science, (e) increased adult literacy, (f) decreased dmgs 
and violence on campuses, (g) providing opportunities for professional development, and 
(h) boosting parental involvement. 

The model of professional development used by the researcher’s state department 
of education, with an emphasis on professional development, was developed by Joyce 
and Showers (1982). The model, which included demonstration, practice, feedback, and 
follow-up, shared many of the same attributes as the state’s professional-learning model. 
The Joyce and Showers model included extensive training and frequent follow-up that 
were often left out of many professional-development programs, which was the case at 
this study site. For professional development to have the intended impact, Joyce and 
Shower’s model provided a substantial framework from which to operate (Steele, 2007). 

The Georgia Department of Education (2007a) embraced the directives of Goals 
2000, along with the creation of their own mandates for local school agencies. Although 
suggested models for professional development were offered, each school was ultimately 
responsible for establishing a forum for professional development. However, there were 
no guidelines regarding the value and purpose behind professional development. 
Evaluation Model 

The Kirkpatrick (1996) evaluation model has been in use for training evaluations 
since the 1950s. The model uses four levels of evaluation to evaluate the effectiveness of 
training: (a) reaction, (b) learning, (c) behavior, and (d) results. According to Naugle 
(2000), “Kirkpatrick’s four-level evaluation model used to measure the effectiveness of a 
teacher or educational system is suggested as the framework to assess and revise teacher 
performance” (p. 31). 

Kirkpatrick (as cited in Naugle, 2000) described Level 1 (reaction) as “how 
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participants feel about a variety of segments in the training, and their perceptions or 
feelings (positive or negative) about the instructor, the material, and the overall 
experience” (p. 4). The idea is if participants do not enjoy or value the experience, the 
rest of the training is pointless. By evaluating the initial or reaction step, evaluators know 
if they can move forward with the program or if they need to reevaluate and again try to 
implement the program with another approach. Naugle stated, “Kirkpatrick noted that at 
this level of evaluation, you are not attempting to measure any degree of learning and the 
usual manner of assessment is a self report from participants” (p. 6). 

In Level 2 (learning), participants are assessed on the skills or knowledge that 
they have gained from participating in staff-development sessions. Through interviews 
and observations, as well as reflective journals, an evaluator can judge to what extent 
participants have improved on the skill or knowledge to be learned. In some cases, pretest 
and posttest surveys have been used to measure more accurately the change in skills or 
knowledge (Naugle, 2000). 

Level 3 of the Kirkpatrick (1996) model involves behavior. Kirkpatrick noted that 
the purpose of using the third level of evaluation was to “measure the extent to which 
participants change their on-the-job behavior” (Naugle, 2000, p. 11). Level 3 involves 
transfer training or asking if what was learned is being used in the workplace. The 
evaluator is investigating whether participants have grasped the concepts and skills from 
the staff-development sessions and if they able to implement new skills and knowledge in 
the workplace. Evaluators are also looking at the relevance of the change and 
sustainability of change. 

Level 4 of the Kirkpatrick (1996) training-evaluation model measures results of 
the training. Naugle (2000) summarized this stage as the formation of a basis of learning 
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upon which to build, the development of skills applied to participants’ learning, and the 
life acquisition of skills to make personal improvements. Measurements of the program’s 
success in Level 4 are completed with posttest surveys, observations, and journal 
reflections (Naugle, 2000). 

Summary 

Through a team’s collaborative efforts, student success and sustainable reform is 
possible. Research has provided evidence and insights indicating that collaboration is a 
process with stumbling blocks. Although published articles allow researchers to find 
ways to overcome the struggles of collaboration, studies have also focused on what 
causes a lack of collaboration. Using protocols represents one method that has a 
substantial impact on a team’s ability to collaborate effectively. 

In any team’ s effort to improve on collaboration, it is important to understand 
what effective collaboration looks and sounds like. With the implementation of 
professional development, members of a team need to have the opportunity to fully 
understand the purpose and benefits of professional collaboration. They also need the 
opportunity to identify specific behaviors of those effectively collaborating. The Georgia 
Department of Education (2007a) designed a set of specific guidelines to which all 
schools were required to adhere in the implementation and use of professional 
collaboration. 

The Kirkpatrick (1996) model allowed for a research study that was both 
qualitative and quantitative. The four levels on which the training program was evaluated 
included reaction, learning, behavior, and results. Each level represented a step closer to 
evaluating and understanding the significance of what was being studied. 
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Chapter 3: Methodology 

Introduction 

The purpose of this study was to evaluate the use of a protocol for organizing and 
implementing effective leadership meetings. This chapter contains detailed information 
about the methodology for the study. Topics include a description of the research design, 
participants, instruments, and procedures of the study. 

Research Design 

In this study, the researcher implemented procedures for evaluating the use of a 
protocol in organizing and implementing effective leadership meetings. A single-group 
research design utilized both quantitative and qualitative measures, making this a mixed- 
methods evaluation. The CCCTAR protocol (RESA, 2002), which was used by many 
professional collaborative groups in the researcher’s state, was implemented. The 
Kirkpatrick (1996) model of evaluation, around which this study was framed, allowed for 
the mixed-methods approach as described earlier. This model allowed for evaluation of 
the participants’ reactions to the professional development, the skills and knowledge 
learned that caused changes in behavior, and the overall results of the professional 
development. The research questions that guided this study are listed below, along with 
the instruments used to collect the data to answer them. 

Research Questions 

The researcher established the following research questions to guide this applied 
dissertation: 

1. What are the leadership team members’ reactions to the protocol as a 
professional-development tool? This was measured using the PDES (Steele, 2007), which 
was completed after the two training sessions. Research Question 1 addressed reaction, 
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which is Level 1 of the Kirkpatrick (1996) model. 

2. What skills and knowledge related to organizing and implementing effective 
leadership team meetings will the leadership team members learn as a result of using the 
protocol as a professional development tool? This was measured using observations of 
the leadership team meetings, responses in reflective journals, and the rubric for Elements 
of an Effective Collaborative Group (Danielson, 2002). Research Question 2 addressed 
learning, which is Level 2 of the Kirkpatrick (1996) model. 

3. Will leadership team members routinely and consistently apply the new skills 
and knowledge related to organizing and implementing effective leadership team 
meetings in their meetings? This was measured using observations of the leadership team 
meetings and responses shared in the reflective journals. Research Question 3 addressed 
behavior, which is Level 3 of the Kirkpatrick (1996) model. 

4. What result will learning the protocol have on participants’ judgments about 
organizing and implementing effective leadership meetings? This was measured using 
pretest and posttest surveys that addressed members’ perceptions of roles and 
professional-collaboration abilities experienced during meetings. Research Question 4 
addressed results, which is Level 4 of the Kirkpatrick (1996) model. 

Participants 

The participants in this study were those elected representatives of each 
department and the administrative staff. All participants were over the age of 25 years. 
There were 1 1 total members on the leadership team. Three of the members were male, 
and eight were female. Lour were at the administrative level, and seven were teachers. 
Three teachers represented different curriculum areas (e.g., language arts, math, science, 
and social studies), one teacher represented special-interest subjects (e.g., consumer 
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science, band, chorus, foreign language, and computer), one teacher represented the 
Physical Education Department, one teacher represented the Special Education 
Department, and one teacher represented the English Speakers of Other Languages 
Department. Administrators included the school’s Instructional Lead Teacher, two 
assistant principals, and the school principal. 

Instruments 

In an effort to conduct a reliable, valid, and fair study, the researcher used the 
following data-collection instruments to measure the results: (a) the Professional 
Learning Community Assessment (PLCA; Huffman & Hipp, 2003), (b) the PDES 
(Steele, 2007), (c) the rubric for Elements of an Effective Collaborative Group 
(Danielson, 2002), and (d) entries in reflective journals. 

Professional learning community assessment (PCLA). This initial measurement 
tool was completed to gain background knowledge of the leadership team’s current 
understanding and feelings toward professional collaboration, as demonstrated at past 
leadership team meetings. The purpose of the PLCA (Huffman & Hipp, 2003) was to 
encourage the participants to think about and evaluate their perceptions of professional 
collaboration (see Appendix B). Five primary areas are addressed in the survey: (a) 
supportive and shared leadership, (b) common values and vision, (c) collective learning 
and application, (d) shared personal practice, and (e) supportive conditions. 

The survey was used with the general permission of Huffman and Hipp (2003), 
who indicated, “The PLCA is available for dissemination and use by educators and others 
as an assessment tool that measures practice observed at the school level relating to the 
five dimensions of professional learning communities and their critical attributes” (p. 74). 
The authors indicated that the instrument could be a very useful tool in assessing 
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perceptions based on the five dimensions of the PLCA (Huffman & Hipp, 2003). 

The PLCA (Huffman & Hipp, 2003) was used because the results of the survey 
show how teachers perceived professional collaboration in their work environment at the 
time that the survey was administered. The purpose of the survey was to assess 
participants’ perceptions of the knowledge and skills needed for collaboration to be 
effective. The surveys allowed the researcher to make inferences about the attitudes of 
the leadership team members regarding how they perceived their abilities to 
professionally collaborate and share in the decision-making processes at the target school. 

The field test for this survey, conducted by Huffman and Hipp (2003), resulted in 
247 completed surveys. The authors indicated, “Descriptive statistics included minimum 
and maximum values of 1 and 4, item means, and standards deviation” (p. 73). The 
method selected to provide construct validity was factor analysis. According to Huffman 
and Hipp, “the analysis series of statistical procedures for the total sample was n = 247. 
Factor identification consisted of the five dimensions of professional-learning 
communities” (p. 74). 

Cronbach’s alpha internal-consistency reliability coefficients were figured for the 
factor subscales of the measure (Huffman & Hipp, 2003). According to the authors, “for 
the five factored subscales, the alpha coefficients ranged from a low .83 (collective 
learning and application and supportive conditions for relationships and structures) to a 
high .93 (shared values and vision). Thus, the instrument yielded satisfactory internal- 
consistency reliability for factor subscales” (p. 74). 

Professional development evaluation survey (PDES). The PDES (Steele, 2007) 
was administered after the first staff-development training session (see Appendix C). 
Participants were asked to respond to seven questions that prompted them to make 
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judgments about the training session. The responses on the PDES (Steele, 2007) 
evaluated Level 1 (reaction) of Kirkpatrick’s evaluation model. The PDES (Steele, 2007) 
had three open-ended questions and four comments to be rated on a Likert-type scale. 

The PDES (Steele, 2007), which was conducted with permission from the author 
(see Appendix D), had been used in a similar applied research study that implemented 
and evaluated teacher study groups to support change in teacher behaviors. Steele found 
validity in the survey as the questions helped to determine the reactions and perceptions 
of the participants regarding the staff development’s focus. The categories reported on 
included new ideas, concepts about which more information was desired, and reactions to 
the staff-development training session. These questions allowed the researcher to 
evaluate the need for additional staff development before moving on to the use of the 
CCCTAR protocol (RESA, 2002). 

Rubric for elements of an effective collaborative group. Through observations and 
use of the rubric for Elements of an Effective Collaborative Group (Danielson, 2002), the 
researcher evaluated team members’ professional-collaboration behaviors (see Appendix 
E). This published rubric assisted in the researcher’s evaluation of the participants’ 
proficiency, as measured by their knowledge and application of behaviors necessary for 
organizing and implementing effective leadership team meetings. Data noted from the 
rubric (Danielson, 2002) informed the researcher if additional professional development 
was needed to ensure the appropriate use of the chosen protocol. If there were such a 
need, the researcher and the lead teacher would have developed additional training 
materials to obtain a deeper understanding of effective meeting behaviors on the part of 
all participants. The elements addressed on the rubric included relationships with 
colleagues, shared and supportive leadership, communication skills, protocol and 
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documentation, and member participation. The rubric represented a continuum in which 
the responses were measured using a Likert-scale item analysis. 

The rubric for Elements of an Effective Collaborative Group was adapted from 
those created by Danielson (2002), which were designed to help educators examine their 
own approach to school organization. Although no statistical data were associated with 
the effectiveness of the rubrics, there was an explanation that the results based on the 
rubrics’ scoring were to help determine where changes needed to be made with in the 
school organization. 

Reflective journals. The participants were provided reflective journals to keep 
throughout the study. The journal entries documented each participant’s feelings and 
attitudes toward personal change in behavior (see Appendix F). Journaling, as defined by 
Gall, Gall, and Borg (2003), represents a process by which participants step back from a 
problematic world and ponder and share ideas about meaning, value, and impact from 
their practice. The journals provided an opportunity for participants to gain insights into 
their own strengths and weaknesses in their current practices. 

Procedures 

This study took place over a period of 7 months. The first 2 months involved 
professional-development sessions and a simulation with the leadership team. The 
leadership team was instructed in the use and benefits of using a protocol and 
professional-collaboration behaviors. The subsequent 5 months involved the 
implementation of the protocol and evaluating its impact on organizing and implementing 
effective leadership meetings. 

Month 1. Prior to the first staff-development session, the PLCA (Huffman & 

Hipp, 2003) was given to each member of the leadership team to gauge individual 
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perceptions of professional collaboration. Because the principal was one of the leadership 
team members in this research study, she was also administered all research instruments. 
Huffman and Hipp considered principals to be coleamers who modeled the “the level of 
learning expected from the professional staff’ (p. 14). The chosen survey published by 
Huffman and Hipp allowed the principal of this target school to be better aware of her 
effectiveness with the faculty at the target school. In an effort to function as an 
operational leadership team, as deemed desirable by the state department of education, 
the principal’s perceived role as the school leader needed to change. Huffman and Hipp 
described this as follows: 

Principals are not coercive or controlling, but seek to share power and distribute 
leadership among staff. In turn, staff increasingly becomes open to changing roles 
and responsibilities. Principals let go of power and nurture the human side and 
expertise of the entire school community. Shared responsibility is apparent 
through broad-based decision making that reflects commitment and 
accountability, (p. 38) 

The PLCA survey allowed the researcher to make inferences about the attitudes of 
the leadership team members, as they perceived their abilities to professionally 
collaborate and share in the decision making at the target school for the 2008-2009 school 
year (Huffman & Hipp, 2003). The same survey was administered at the conclusion of 
the study. A comparison of the responses was made to aid in determining the outcome 
results of this study. The average score for each item from the PLCA pretest and posttest 
survey was reported in table format for the entire group of participants. The data were 
then disaggregated by individual sections, including (a) shared leadership and supportive 
leadership, (b) shared values and vision, (c) collective learning and application, (d) 
shared personal practice, and (e) supportive conditions. 

The researcher also analyzed the data for each item within each section to 




35 



determine trends and patterns. It was the intent of this researcher to measure significant 
changes in the responses after staff development had taken place and the protocol had 
been used as the structure of the leadership team meetings for 7 months. Likert-scale item 
analysis was utilized in comparing the PLCA results to measure growth, sustainability, or 
diminished views of the leadership team members’ effectiveness in shared decision 
making and collaboration in the target school. 

The target school’s instructional lead teacher facilitated the two training sessions 
and the simulation. He utilized action learning and reflection discussion. In action 
learning, students are engaged in the learning process and are more likely to grasp what is 
being taught. Revans (as cited in BNET, 2008) pioneered action learning as a 
management and organizational development tool. It was based on the principle that 
people learn best when they focus on a problem together (BNET, 2008). The lead teacher 
presented the information (see Appendixes G and H) to the leadership team members 
with the understanding that the members could ask questions, share experiences related to 
the topics, and participate in the lesson with free will. 

The agenda for the first staff development (see Appendix G) contained the 
specific information that was presented. Action learning was the strategy used for the first 
professional-development session. In that session, the lead teacher of the target school 
discussed and presented the CCCTAR protocol’s purpose and benefits. He then described 
and validated the use of reflective journals. Action learning was utilized for the structure 
providing a less formal situation. In the first training session, the lead teacher was 
presenting information, but he allowed for teachers’ personal experiences with protocols 
to help enlighten the group. At the conclusion of the first staff development, the lead 
teacher asked the leadership team members to appraise the staff-development session’s 
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key points related to the CCCTAR protocol’s purpose and use of reflective journals. 
Leadership team members completed and submitted the PDES at the conclusion of the 
staff development. Members were asked to state in their own words the session’s main 
points and rate their own understanding of key concepts shared in the session. 

The goals of the first professional-development training sessions were measured, 
as participants demonstrated their understanding of the protocol’s use and benefits with 
their ability to transfer their learning in utilizing the protocol effectively during their 
monthly meetings. Leadership team members were asked to express their learning from 
the staff-development training session as to the use and benefit of the reflective journals 
in their composition of their responses throughout the study. The reflective journals were 
administered to all participants at the conclusions of the second staff-development 
training session. In addition to the explanation of the purpose and significance of the 
responses in the journal, participants were asked to add comments to their journals in 
regards to their thoughts on the effectiveness of collaboration among team members. 
They were encouraged to be truthful and open in their frustrations, challenges, and 
success. 

Journals were numbered but not assigned. Each member chose a number and 
maintained that numbered journal throughout the study. Members had 1 week to add to 
their journals after the conclusion of each monthly leadership meeting. The journals were 
placed on a table for members as they entered each leadership team meeting, thereby 
allowing for anonymity in the retrieving of journals. The journals were then turned in by 
each member of the leadership team to a box set aside in the office of the counselor’s 
secretary. The researcher collected the journals from the secretary’s office 1 week after 
scheduled leadership team meetings. An analysis of the journal entries was presented in a 
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narrative summary documenting the outcomes of participants’ reactions to each 
leadership meeting. The researcher made notes on commonalities among entries as to the 
group’s ability to gain insight of the collaborative process. They were collected and 
analyzed each month throughout the research study. 

Month 2. The agenda for the second staff-development session (see Appendix H) 
contained the specific information shared with the leadership team. The lead teacher 
demonstrated and practiced the steps of the CCCTAR protocol (RESA, 2002) utilizing 
the reflection learning strategy, which is associated with constructivist learning theory, 
where learning is based on the experience with the subject matter. The lead teacher 
conducted the second staff development using the protocol’s steps as his guide in the 
implementation of the meeting. The idea was that the leadership team members would 
experience the protocol as they learned about it. They participated in an organized 
meeting as practice. The lead teacher demonstrated his expectations of an organized and 
effective leadership committee in the way he conducted the second training session. 

Members experienced the segmented periods time of the protocol as they learned 
about the CCCTAR protocol (RESA, 2002). Time limits were not used in this training 
session. Segments 1 and 2 of the protocol involved the sharing of the agenda and 
effective professional-collaboration behaviors. Segments 3 through 5 of the protocol 
followed the session on professional collaboration. 

The lead teacher practiced action learning during his definitions of effective 
behaviors of professional collaboration. The idea of empowering each member of the 
leadership team to share ideas, without fear of judgment, moved the group into the 
specific behaviors of effective collaboration. Marzano (2003) established three 
characteristics of importance in collaborative behaviors: (a) optimism, (b) honesty, and 
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(c) consideration. Optimism increases teachers’ self-esteem and motivation. According to 
Marzano, “honesty is characterized by truthfulness and consistency between words and 
actions” (p. 177). Consideration “is sometimes referred to as a people orientation or a 
concern for people” (p. 178). Honesty and consideration both help build interpersonal 
professional relationships. These were the behavioral characteristics sought from each 
leadership team member, in addition to democratic decision making, constructive 
feedback, support of colleagues, active participation, maintaining of an open mind, and 
leadership roles. The researcher anticipated that with the establishment of such 
characteristics, effective leadership meetings would take place. The journals were handed 
out at the conclusion of the second training session. 

The first journal entry assessed the participants’ understanding of professional 
collaboration. The directions for discussions of Journal Entry 1 were as follows: 

1. In your own words, what are some of the appropriate behaviors demonstrated 
by those who collaborate effectively on a professional level? 

2. Do you find any similar behaviors used in your leadership meeting either by 
yourself or peers? 

3. What behaviors could you begin to develop that might enable the leadership 
group to functional operationally? 

The participants practiced these collaborative attributes in a simulation hosted by 
the lead teacher (see Appendix I). This served as the third training session. In that 
simulation, the teachers were presented with the challenge of rectifying a new policy on 
discipline. Prior to the simulation, teachers and administrators were asked to think about 
and come up with suggestions for improving schoolwide discipline procedures (for 
simulation purposes only). The leadership committee gathered for a period of 70 minutes. 
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A timer was kept that marked the transitions of the protocol segments. The lead teacher 
opened the simulation with an overview of the agenda and stated that the goal of the 
meeting was to come up with three ideas for a new schoolwide discipline policy. His 
presentation style optimized the characteristics of professional collaboration (i.e., 
optimism, honesty, and consideration). 

This topic of discipline was chosen because of the vested interest of each 
leadership committee member in its outcome. The participants were given the rubric for 
Elements of an Effective Collaborative Group (Danielson, 2002) and asked to circle 
where they think the group would score. They then revisited the rubric at the end of the 
meeting and rescored the group. The group discussed their anticipated results with the 
actual results in the final 10 minutes of the simulations. 

The lead teacher moved into his presentation of the material and followed it with 
a 30-minute discussion period, during which effective collaboration behaviors were 
expected of the participants. The researcher anticipated that the lead teacher might have 
had to redirect certain comments and attitudes to demonstrate effective collaboration. He 
modeled his own questions and comments with characteristic of effective collaboration 
(e.g., open mindedness, constructive feedback given, support of colleagues’ thoughts and 
opinions, participation, and active listening). 

The leadership committee then reevaluated the meeting using the rubric as a guide 
to score how they felt that the group actually collaborated. They then journaled their 
experience. This did not occur in their actual journals but on the back of the rubric 
handout. During the final 5 minutes in the protocol, the lead teacher reviewed key points 
of the discussion and presented the three new ideas for a discipline policy. 

The additional 10 minutes slotted in this simulation were for reviewing how the 
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group felt that they had collaborated. Discussions of the rubric scorings occurred. This 
session was designed to clarify behaviors and misconceptions, and serve as an example of 
positive and negative behaviors for reflection, which was ideal in instilling new 
behaviors. 

The first two sessions provided data for an evaluation of Research Questions 1 
and 2 of this study, which utilized Level 1 (reaction) and Level 2 (learning) of the 
Kirkpatrick (1996) model. Level 2 was continuously evaluated through reflective journals 
and observations. If the researcher had found frustration or a lack of understanding on 
behalf of the participants, additional staff development would have been made available. 
The instruction would have focused on the areas of need deemed by participants’ 
responses in their journals and researchers observations of participants’ reactions in 
sessions. The researcher would have provided the additional staff development. Naugle 
(2000) shared that at this second level, it was logical to move into a more formalized 
evaluation process, hence the use of the rubric for Elements of an Effective Collaborative 
Group (Danielson, 2002). 

Months 3-7. Leadership team members met once a month for a total of 7 months. 
Inside each journal were the steps of the chosen protocol. The principal served as the 
facilitator for these leadership team meetings. Through observations and use of the rubric 
for Elements of an Effective Collaborative Group (Danielson, 2002), the researcher 
evaluated team members’ professional-collaboration behaviors (e.g., open-mindedness, 
constructive feedback, support of colleagues’ thoughts and opinions, participation, and 
active listening). This addressed Research Question 3 of this study and utilized Level 3 
(behavior) of the Kirkpatrick (1996) model. Kirkpatrick stated that evaluation at Level 3 
revolves around the need to appraise behavior using statistical analysis in addition to 
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anecdotal reports (Naugle, 2000). The researcher analyzed participants’ skills and 
knowledge based on the data collected using the rubric. With the data collected, the 
researcher measured the leadership team members’ ability to organize and implement an 
effective leadership meeting. 

As part of the evaluation of the first three levels of the Kirkpatrick (1996) model, 
the researcher analyzed the pretraining PLCA (Huffman & Hipp, 2003) using an item 
analysis. The researcher was looking for trends in the averages regarding how the 
leadership team members viewed professional collaboration as it applied to the leadership 
team’s effectiveness in the target school. The researcher analyzed the first two journal 
entries by looking for data that supported the need for additional staff development on 
protocol use and collaboration skills. Anecdotal notes were taken during training sessions 
and leadership meetings. These notes aided the researcher in observing trends and 
patterns in the areas of effective collaboration, protocol use, and active participation. 

After the two training sessions and 5 months of using the CCCTAR protocol 
(RESA, 2002), the researcher presented the leadership team members with the 
posttraining PLCA (Huffman & Hipp, 2003) to measure Level 4 (results) of the 
Kirkpatrick (1996) evaluation model. These surveys, along with the observational data 
collected by the researcher, were used to evaluate participants’ ability to organize and 
implement an effective leadership meeting. A Likert-scale item analysis was made use of 
in comparing the PLCA results to measure growth, sustainability, or diminished views of 
the leadership team members’ effectiveness in collaboration at the target school. In Level 
4 of the Kirkpatrick model, behavior or performance change should be evaluated after at 
least 3 months, allowing the participants an opportunity to put into practice what they 



have learned (Naugle, 2000). 
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The researcher chose the CCCTAR protocol (RESA, 2002) because many 
professional study groups in the researcher’s state had used it. The RESA was one of the 
local state agencies that supported teacher learning at the target school. This protocol 
helped professional educators develop and enhance attitudes, mindsets, and skills that 
would allow them to become effective in professional collaboration. The process 
represented a relatively risk-free way to get authentic feedback on topics of discussion. 
Steele (2007) wrote that the protocol was “derived from the Coalition of Essential 
Schools by Allen and McDonald (as citied in Allen, Blythe, & Powell, 1996)” (p. 25). 
Those authors praised the protocol’s process as one that prevented attacks and rebuttals 
and prevented the participants from being on the defensive. 

The GAPSS was a 3-year process, and the target school was required to send in 
updates twice a year to the state department of education in regard to the improvements 
being made based on the GAPSS team’s recommendations. This study served as evidence 
that the concerns for collaboration of the leadership team had been addressed. It was 
anticipated that the learning and implementation of a protocol would enable the 
leadership team to gain a capacity for immediate change in their professional- 
collaboration skills. With effective professional-collaboration skills, the leadership team 
of the target school could be rated as operational, according to the GAPSS standards. 

In a collaborative model developed by the New Jersey Hospital Association 
(2008), the time frame of 6 to 13 months was recommended to instill effective change. 
The assumption was made that this time period was also viable for this collaborative 
study. The Kirkpatrick (1996) evaluation model suggested that a time period of no less 
than 3 months be used to implement a measurable change in behavior (Naugle, 2000). 
This study was implemented with the leadership team for 7 months. 
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This study may have served as a catalyst for change within the target school. The 
leadership team noted success with the use of a protocol, and other curriculum and 
professional-learning groups within the target school now had an incentive to utili z e 
protocols in their meetings as well. According to McDonald et al. (2007), “whenever talk 
has important consequences, we deserve the chance to think through what we want to say 
and an environment where what we choose to say can be heard and respected” (p. xiii). 
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Chapter 4: Results 

Introduction 

This chapter presents a statistical analysis of the findings used to answer each 
research questions. The numerical data are accompanied by an analysis of the journal 
entries shared by the participants. Together, these two types of data (quantitative and 
qualitative) provide a concise record of the findings for this study in seeking to evaluate 
the effects of the use of a protocol on implementing and organizing effective leadership 
meetings. 

Results for Research Question 1 

The first research question asked, “What are the leadership members’ reactions to 
the protocol as a professional-development tool?” This was measured by the data 
collected from the PDES (Steele, 2007) that was administered after the second staff 
development training session. Research Question 1 addressed Level 1, Reaction, of the 
Kirkpatrick (1996) model. 

The PDES (Steele, 2007) showed nearly universal positive responses in regard to 
the professional development. Table 1 shows how the 11 participants rated their 
experience across four measures of effectiveness. Answers ranged from strongly agree to 
strongly disagree. Ninety-one percent of the 44 total responses indicated that the 
participants either strongly agreed or agreed that the workshop was clear, augmented 
their understanding of protocols, improved their collaborative skill sets, and assisted them 
in the use of a protocol to improve leadership (see Appendix K). 

Comments shared on the PDES (Steele, 2007) indicated an understanding of the 
intended purpose of the staff development. In response to the question that addressed new 
concepts, 7 of the 1 1 participants reported that the protocol represented a newly 
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prescribed format with which they were unfamiliar. One member stated, “I never heard of 
having steps in conducting a meeting with adults.” 

Table 1 



Protocol Ratings on Suri’ey 



Survey item 


SA 


A 


N 


D 


SD 


Staff development objectives were clearly 
identified and accomplished. 


7 


4 


0 


0 


0 


I have more knowledge and understanding 
of what a protocol is. 


5 


6 


0 


0 


0 


I learned specific skills needed in effective 
professional collaboration. 


4 


6 


1 


0 


0 


I believe the use of a protocol will help our 
leadership team function as operational. 


4 


4 


2 


1 


0 



Another member wrote, “Protocols are the buzz word of late; glad to get some 
experience with one.” Two participants wrote that they had not made the connection prior 
to staff development on the importance of collaboration in the leadership meetings. 
Another participant reported, “I figured we were collaborating in our conversing. I now 
see the difference between the two.” Two participants chose not to write any comments. 
Responding to the prompt, I would like to know more about the following concepts, only 
one participant asked for more information on Marzano’s (2003) book. She asked for a 
copy of the text or where she could locate the text on her own for further reading. 

Results for Research Question 2 

The second research question asked, “What skills and knowledge related to 
organizing and implementing effective leadership team meetings will the leadership team 
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members learn as a result of using the protocol as a professional-development tool?” 
Observational data of the leadership team meetings, recorded on the rubric for Elements 
of an Effective Collaborative Group (Danielson, 2002), and comments in reflective 
journals were used to address this question. Research Question 2 addressed Level 2, 
Learning, of the Kirkpatrick (1996) model. Table 2 presents the aggregated findings of 
the six observed meetings. The level of proficiency using the rubric criteria in the table 
indicated a numerical progression toward mastery. The leadership team meetings were 
assessed according to the following elements: (a) relationships with colleagues, (b) 
shared and supportive leadership, (c) communication skills, (d) protocol and 
documentation, and (e) leadership team member participation. Answers ranged from 1 
{needs work ) to 4 ( mastered f). 

Table 2 



Evaluation of Meetings According to Rubric of Elements 









Meeting 






Item 


1 


2 


3 


4 


5 


6 


Relationship with colleagues 


1 


2 


2 


3 


3 


3 


Shared and supportive leadership 


1 


2 


3 


2 


3 


3 


Communication skills 


1 


2 


2 


3 


3 


3 


Protocol and documentation 


3 


3 


4 


4 


4 


4 


Leadership team member 
participation 


2 


2 


2 


3 


2 


2 



The first element addressed the relationships of leadership team members with 
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colleagues. During Month 2 of the study, the leadership team members had not used 
collaboration. One member of the leadership team noted eye rolling and frustrated 
whispers at some of the newer members’ contributions. Two members shared their 
frustration about their own inability to add to the conversation due to a lack of knowledge 
on the topics discussed. The group did have a friendly climate, but there was no 
measurable sense of camaraderie. 

During Months 3 through 6, the leadership team slowly moved toward a 
proficient level for mastery learning for relationships with colleagues. Journal entries did 
not indicate any significant factors preventing a quicker progression from needs work to 
proficient in mastering relationships with colleagues. One participant wrote, “I’m not 
completely comfortable sharing my thoughts in these meetings. I really don’t know the 
other members well.” Another member shared, “I think it would be helpful to serve on 
the team for a period of no less than 2 years, so that we become comfortable and trusting 
of each other.” 

By Month 7 of the study, the leadership team had operated at the proficient level 
for three consecutive meetings, based on the rubric for Elements of an Effective 
Collaborative Group (Danielson, 2002), in connection with relationships with colleagues. 
Although a level of comfort and trust was slowly built throughout the 7 months, it was 
not mastered. In journaling, a member shared how great improvements had been made in 
the relationships among the team in the following statement: “I feel I have gained a new 
set of colleagues outside my curriculum group.” A leadership team member reported that 
for the team to become truly operational (Georgia Department of Education, 2007b), the 
team could not be changed each year. One participant stated, “The changing of members 
each year defeats the purpose of staff development sessions and the collaborative 
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relationships that have taken time to be built.” Another leadership team member felt more 
a part of this team after several months of serving on it, but he did not really feel that 
personal opinions mattered to the group as a whole. He reported, “Not being a classroom 
teacher makes me feel as an outsider and that my opinions are not valid.” A different 
member acknowledged that the group as a team had come a long way since February. He 
wrote, “Being made aware of physical responses and not just verbal responses has made 
us a better functioning school leadership team.” 

Shared and supported leadership rankings fluctuated based on the posted agenda 
goals (see Table 2). Meeting 4 was focused on the principal sharing information rather 
than having a discussion. The leadership team members showed a decline in their ability 
to achieve mastery of shared and supportive leadership in this meeting. To prevent this 
from occurring in the future, the principal sent out detailed agendas for the next two 
meetings. Attached to the agendas was important supporting information. The 
information was to be read prior to the meeting so time spent in the meeting could be 
used productively to discuss the topics rather than just to review them. This enabled the 
leadership team to progress in their move toward mastery of shared and supportive 
leadership during Meetings 5 and 6. 

In reviewing the journal entries, factors preventing the leadership team from 
mastering shared and supportive leadership could be identified. Newer leadership team 
members reported their lack of knowledge about their role within the leadership team. 
They believed that they were not prepared to make significant judgments about some of 
the topics discussed in the meetings. One member stated, “I fear retribution from the 
principal if I shared my actual opinions.” Also shared in journaling were complaints 
about several members who dominated the conversations. According to one participant, 
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“I have things I’d like to say, but every time I open my mouth, someone is already 
speaking; by the time they finish, my point seems moot or we’ve run out of time.” 
Another participant stated, “Member 7 always has something to say. She dominates the 
time allotted for discussion.” Less experienced members were intimidated in speaking up 
against those members serving on the leadership team prior to the study. One participant 
reported, “I’m new to the team and the school. This year is my learning curve. I’m here to 
listen and learn.” 

The staff-development training sessions focused on the leadership team’s ability 
to collaborate effectively. Communication skills addressed on the rubric for Elements of 
an Effective Collaborative Group (Danielson, 2002) showed a steady progression toward 
mastery. The first journal entry after the second staff-development session asked 
members to evaluate their own understanding of collaborative behavior. Many leadership 
team members merely restated what had been taught in the first staff development, but 
some leadership team members elaborated on what they personally needed to do to better 
their communication skills. One member declared, “Participate”! A separate member 
shared, “I need to be more self-assertive, now knowing how to collaborate, and it should 
be more comfortable for me.” Another member wrote, “Saying what was on our minds 
when walking away from the meeting is a waste of everyone’s time.” Another leadership 
team member reported that the leadership team members needed to stay focused on the 
subject of discussion instead of discussing unrelated side topics that lead away from the 
main focus of the meeting. This sentiment was shared in the early journal entries (1 and 
2), but it did not resurface as the study progressed. 

Communication began to improve by Month 5 of the study. The leadership team 
moved from sufficient mastery of communication skills to proficient mastery of 
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communication skills, based on data recorded on the rubric for Elements of an Effective 
Collaborative Group (Danielson, 2002). The growth in the leadership team’s abilities in 
communication was noted in the journal entries of the participants. The discussion for 
Journal Entry 4 asked, “Does the use of the protocol seem to keep the group on focus and 
allow for optimal professional collaboration?” Members responded with a resounding 
yes, indicating THAT the protocol allowed for a set time to participate (i.e., collaborate). 
In the journal entries, one member wrote about how the protocol established a sense of 
professionalism. Another journal entry mentioned how the leadership team member was 
skeptical at first as to what a protocol could do. He thought following a protocol would 
interfere with the natural flow of conversation; however, he had since learned that the 
protocol actually promoted professional collaboration. He continued by commenting on 
how the protocol kept the team focused on specific times to speak and allowed for an 
efficient meeting and not just random conversations. 

A factor observed that thwarted mastery of collaboration during leadership team 
meetings was the domination of several members over all conversations. Some members 
overparticipated while others held their comments until the point of reference had passed. 
In responding to another journal prompt (What needs to occur before the leadership team 
can become fully operational in an effective leadership committee?), leadership team 
members’ comments in their journal entries confirmed the same observations. One 
member noted that she was more comfortable taking notes because other members spoke 
the whole time. Another leadership team member declared, “The leadership team would 
not ever be fully operational.” The stated reason was that “not all of the members were 
professional or considerate enough for the team to be fully operational.” 

A newer member felt out of the loop and for the leadership team to be rated as 
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operational, all members need to be included and heard. The uncomfortable newer 
member went on to say it would never be possible to comfortable if the members kept 
changing each year. An opposing opinion came from another member’s journal. The 
member thought the protocol had the leadership team well on their way to being fully 
operational. She felt the meetings were far more productive than in years past, as well as 
more focused. She wrote that she now “walks away from the meetings with a sense of 
accomplishment.” Another member noted that the procedures this year were more 
professional. They complimented the efforts of researcher and the influence of the 
protocol over the functioning of the meetings. 

Protocol and documentation were used proficiently and later mastered as a result 
of professional-development sessions. The steps for the protocol were visible at each of 
the observed meetings, thereby allowing for an awareness of each step. The principal 
verbally acknowledged each step as the group progressed through the CCCTAR protocol 
(RESA, 2002). 

The leadership team progressed to the mastery level of learning in protocol and 
documentation, as rated on the rubric for Elements of an Effective Collaborative Group 
(Danielson, 2002) by Month 4. The factors preventing immediate mastery of protocol and 
documentation were the forgetting of steps or reversing the order of steps. Once 
familiarity in the use of the protocol was established through repetition, mastery became 
evident and was maintained. 

In the area of Leadership Team Membership Participation, there were consistent 
absences of several members’ contribution efforts. The leadership team maintained a 
sufficient rating for the entire study, as indicated by data recorded on the rubric for 
Elements of an Effective Collaborative Group (Danielson, 2002), with the exception of 
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Month 5. The exceptional performance of the leadership team members was due to the 
attendance of a state representative at the leadership team’s monthly meeting. All 
members of the leadership team were engaged in the conversation. Physical and verbal 
responses were documented during the meetings by the researcher. 

The leadership team did not progress to mastery in participation, however, 
because all members did not participate verbally. The rubric for Elements of an Effective 
Collaborative Group (Danielson, 2002) required that all voices be heard for mastery to be 
achieved. Physical responses were not counted toward participation in regard to voices 
heard. 

Results for Research Question 3 

The third research question asked, “Will leadership team members routinely and 
consistently apply the new skills and knowledge related to organizing and implementing 
effective leadership in their team meetings?” Observations of the leadership team 
meetings and the responses shared in the reflective journals constituted the data used to 
answer this research question, which addressed Level 3, Behavior, of the Kirkpatrick 
(1996) model. To assess the leadership team members’ ability to routinely and 
consistently apply collaborative behaviors, specific observations were made during the 
leadership team meetings throughout the study. The researcher evaluated the aptitude of 
participants in their ability to demonstrate the behaviors taught in the second staff- 
development session. 

Participants were randomly assigned numbers in place of their names or titles. 

The numbers were used throughout the study to aggregate the data when looking at 
individual gains in collaborative behaviors. Tables and graphs were used to aid in 
evaluating and documenting leadership team members’ collaborative behaviors in regard 
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to Marzano’s (2003) characteristics of collaborative behaviors. Five of the nine 
collaborative behaviors were measurable though informal observations. The behaviors of 
(a) consideration, (b) support for colleagues, (c) active participation, (d) constructive 
feedback, and (e) serving in leadership roles were observed during meetings and counted. 
The counts of behaviors focused on determining learned behavior (Kirkpatrick, 1996). 

During the second staff-development session, the instructional lead teacher taught 
collaborative behaviors to the leadership team members. The leadership team was taught 
that consideration “is sometimes referred to as a people orientation or a concern for 
people” (Marzano, 2003, p. 178). Support of colleagues occurs when nonjudgmental 
comments and facial reactions are made to others’ comments. There is to be no speaking 
when others are speaking. Active participation is listening and acknowledging with head 
nods and sharing related comments and ideas at appropriate times during a meeting. It is 
also rephrasing a misunderstood comment. Providing constructive feedback occurs when 
disagreeing is essential and nobody makes disagreements personal. The leadership team 
members were taught that if the idea is not similar to their own, they should focus on the 
idea and not the person sharing. The role of leadership should be shared voluntarily. 

Table 3 illustrates the team’s ability to acquire and apply the specific 
collaborative skills over the course of 5 months. During each leadership team meeting, 
the researcher watched and listened for consideration, support for colleagues, active 
participation, constructive feedback, and serving in a leadership role. Leadership team 
members who demonstrated any of these behaviors received a count of 1 for the specific 
behavior. Each month, a graph similar to Table 3 was totaled and then averaged. The 
researcher numerically averaged the tallies to determine the frequency levels of each 
member’s behavior. Frequency levels ranged from 3 ( above average ) to 0 ( undetermined 
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application (i.e., lack of evidence of specified behavior). As a team, if the average score 
were 1.71 (e.g., consideration), this indicated that the team as a group performed between 
below average and average in the area of consideration across the 5 months. 

Table 3 



Mean Obserx’ation Scores by Area 



Area 






Month 






Average 


1 


2 


3 


4 


5 


Consideration 


1.45 


1.45 


1.82 


1.91 


1.91 


1.71 


Support of colleagues 


1.09 


1.64 


2.09 


1.91 


2.00 


1.74 


Active participation 


.91 


1.73 


1.91 


1.82 


1.91 


1.65 


Provide feedback 


.82 


1.18 


1.27 


1.45 


1.64 


1.27 


Leadership role 


.73 


1.09 


1.55 


1.27 


1.55 


1.23 



Table 3 also shows how the team progressed on average in the area of 
consideration, from a low of 1.45 to a high of 1.91, with a plateau appearing in Months 4 
and 5. As a team, the group progressed from 1.09 in Month 1 to a high of 2.00 in Month 
5. The areas of active participation, providing feedback, and leadership all started below 
1.00 and progressed over the 5-month period. Change in the behaviors was calculated by 
subtracting Month 5 (highest) from Month 1 (lowest). The greatest change in the team as 
a whole was in active participation (1.00), and the least amount of change in the team as a 
group was in consideration (.46). 

In addition to scoring the team as a whole across 5 months, the scores of 
individuals were also averaged during the 5 months to illustrate differences in individuals 
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on the leadership team. Table 4 presents the data for this analysis. As the table indicates, 
there was divergence in the scores of individuals for the 5 months as there was 
divergence in their scores for the different behaviors. Individual 1 was, on average, high 
in consideration but quite low in leadership. Individuals 5 and 10 did not demonstrate any 
behaviors and were classified as 0.00 for four of the five behaviors and had average 
scores near 0 for consideration. Individual 4 was high across all behaviors, as was 
Individual 8. Leadership behaviors consistently had the lowest scores, and 6 of the 
participants were below-average application in leadership; however, 2 participants had 
high scores. Scores on average across the 5 months tended to be between average 
application and below-average application for most of the participants. 

Table 4 



Individual Observation Scores by Area 



Individual 


Consideration 


Support 


Participation 


Feedback 


Leadership 


1 


2.40 


1.40 


1.80 


1.80 


.40 


2 


1.00 


1.00 


2.40 


.60 


.60 


3 


2.20 


2.60 


1.00 


0.00 


.40 


4 


2.60 


2.60 


2.40 


2.20 


2.80 


5 


.20 


0.00 


0.00 


0.00 


0.00 


6 


1.80 


2.40 


1.60 


2.00 


1.40 


7 


2.00 


1.60 


2.60 


.80 


2.60 


8 


2.60 


2.60 


2.80 


2.80 


2.80 


9 


2.00 


2.40 


2.20 


2.00 


.80 


10 


.20 


0.00 


0.00 


0.00 


0.00 


11 


1.80 


2.60 


1.40 


1.80 


1.80 



Journal entries from the leadership team members showed discrepancies in 



participants’ abilities to routinely and consistently apply the new skills and knowledge 



related to organizing and implementing effective leadership meetings. One participant 
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stated, “Knowing how to cooperate and to actually cooperate are very different things.” 
Another participant wrote, “Now that I know how to actually collaborate, it should be 
easy with the awareness of the whole group.” 

Journal Entry 2 asked, “Do you feel you and your peers have had enough training 
on what professional collaboration looks and sound like?” Participant 7 responded, “I 
don’t think we need more training. I think ‘professional collaboration’ is a learning 
process; the more we do it, the more comfortable we’ll be with it.” Participant 10 
indicated that professional training was pointless in this type of setting. She stated, “As 
adults, we will just converse and waste time no matter how much we are encouraged to 
follow a protocol.” 

Of the 11 journal entries, only 1 participant suggested a need for additional 
training. That participant wrote, “More training is needed, but there is no time during 
school hours.” Other members reported that they had received sufficient training, and one 
member wrote that the handouts from the staff developments could be used as reference 
pieces in the future. One participant stated, “Now having had formal training on 
collaborative behaviors maybe our group will move to more professional behavior in 
place of the cordial behaviors of the past.” All members agreed that the professional- 
development sessions and the protocol would help the team to work toward reaching an 
operational ranking, as deemed acceptable by the GAPSS reviewers (Georgia Department 
of Education, 2007a). One member of the team reported, “We are making the necessary 
changes needed to improve and become an effective leadership team.” Another member 
shared that the protocol may slow down the collaborative process, but it seems necessary 
to keep the group focused. Participant 9 indicated, “The protocol leaves no room for 
misunderstandings because the decision or discussions are revisited at the end.” 
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Journal Entry 4 asked, “Does the use of the protocol seem to keep the group on 

focus and allow for optimal professional collaboration?” Responses again ranged in the 

members’ reflections and opinions of the group’s efforts to routinely and consistently 

apply the new skills and knowledge. Participant 8 wrote, “I think the group is on task for 

the most part.” Participant 1 shared the following: 

I feel having our protocol establishes a sense of professionalism, even if we don’t 
always maintain it. When we follow the protocol for the order of the meeting, 
things flow much better than when it is not followed. I find myself biting my lip 
and mentally eye rolling rather than acting out those frustrations. I’ve learned the 
collaborative process is just that, a process. I think I’m better for the lessons of 
this year. 

Three other members shared that the protocol keeps the group focused. One 
wrote, “The use of the protocol keeps us professional. I was skeptical at first, but I have 
come to appreciate the protocol.” Another participant wrote, “The protocol seems to 
really keep the leader more focused than the rest, which allows for an effective meeting.” 
Conversely, another participant reported, “We were fine last year, and I don’t 
understand why we need prescribed steps to have a meeting.” Another member shared, 
“I’ve never been on a leadership team before, but this seems demeaning to have steps to 
follow in a meeting with adults. This is something I would do with my students.” 
Participant 5 declared, “I am insulted by having the steps posted.” Two other members 
chose to not respond to this entry. 

The final entry prompted participant to once again reflect on the group’s ability to 
routinely and consistently apply the new skills and knowledge related to organizing and 
implementing effective leadership in their team meetings. The prompt asked, “Have the 
leadership team members improved in their abilities to collaborate effectively with the 
use of a protocol?” Five of the 1 1 members did not respond at all to the final prompt. 
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Participant 7 wrote in bold, “Yes”! She then elaborated, “Some members still have a lot 
of growing to do, but over all this year’ s leadership team meetings were far more 
productive than past meetings.” The other five responses were also positive. None of the 
members truly elaborated on their experience with the protocol, but all were in favor of 
the use of the protocol. 

Results for Research Question 4 

Research Question 4 asked, “What result will learning the protocol have on 
participants’ judgments about organizing and implementing effective leadership 
meetings?” The pretest and posttest surveys that addressed members’ perceptions of roles 
and professional-collaboration abilities experienced during meetings were used to 
measure the effectiveness of the protocol. This question addressed Level 4, Results, of 
the Kirkpatrick (1996) model. 

The researcher conducted a test of statistical significance on the data set to 
determine the result that learning the protocol had on participants’ judgments about 
organizing and implementing effective leadership meetings. Given the ordinal nature of 
the dataset, based on a Likert scale, the Mann- Whitney test was the appropriate statistical 
test. 

The researcher tested the overall metrics as well as six subsections: (a) shared and 
supportive leadership, (b) shared value and vision, (c) collective learning and application, 
(d) shared personal practice, (e) supportive conditions for relationships, and (f) 
supportive conditions for structures. Tables 5, 6, 7, 8, 9, and 10 show the compiled results 
of pretest and posttest survey responses for each subsection. Answers ranged from 4 
(, strongly agree ) to 1 ( strongly disagree). Appendix L presents the details of the statistical 
analyses for each subsection of the survey. The subsections of the survey that showed 
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statistically significant differences from pretest to posttest in participants’ judgments 
were (a) shared and supportive leadership (z = -2.535), (b) shared value and vision (z = 
-2.111), (d) shared personal practice (z. = -2.082), and (f) supportive conditions for 
structures (z = -1.685). The subsections of the survey that did not show statistically 
significant differences from pretest to posttest were (c) collective learning and application 
(z = -1.054) and (e) supportive conditions for relationships (z = -1.315). 

Table 5 



Results for Shared and Supportive Leadership 



Item 




Pretest 






Posttest 




4 


3 


2 


1 


4 


3 


2 


1 


1 


0 


4 


3 


4 


2 


5 


2 


2 


2 


0 


4 


6 


1 


3 


6 


2 


0 


3 


1 


3 


5 


2 


4 


4 


2 


1 


4 


5 


4 


2 


1 


6 


4 


1 


0 


5 


0 


3 


2 


6 


0 


5 


5 


1 


6 


0 


3 


4 


4 


1 


5 


5 


1 


7 


1 


8 


2 


0 


2 


8 


1 


0 


8 


0 


5 


6 


0 


1 


7 


3 


0 


9 


0 


2 


3 


6 


2 


6 


3 


0 


10 


1 


1 


5 


4 


1 


2 


5 


3 


Total 


8 


37 


38 


28 


23 


52 


29 


8 
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Table 6 



Results for Shared Value and Vision 







Pretest 




Posttest 




Item 


4 


3 2 1 


4 


3 2 


1 



1 


0 


7 


3 


1 


1 


8 


2 


0 


2 


2 


6 


1 


2 


3 


7 


0 


1 


3 


0 


3 


4 


4 


3 


4 


4 


0 


4 


2 


7 


2 


0 


4 


7 


0 


0 


5 


0 


3 


3 


5 


5 


4 


2 


0 


6 


3 


5 


3 


0 


1 


7 


3 


0 


7 


3 


8 


0 


0 


3 


8 


0 


0 


8 


0 


6 


3 


2 


3 


6 


2 


0 


Total 


10 


45 


19 


14 


23 


51 


13 


1 



Table 7 

Results for Collective Learning and Application 


Pretest 






Posttest 




Item 4 3 2 


1 


4 


3 2 


1 



1 


2 


8 


1 


0 


3 


7 


1 


0 


2 


3 


6 


2 


0 


6 


5 


1 


0 


3 


0 


4 


2 


5 


5 


4 


2 


0 


4 


0 


3 


4 


4 


0 


6 


2 


3 


5 


1 


7 


3 


1 


2 


6 


3 


0 


6 


6 


5 


0 


0 


7 


4 


0 


0 


7 


2 


4 


3 


2 


2 


6 


3 


0 


8 


7 


4 


0 


0 


6 


5 


0 


0 


Total 


21 


41 


15 


12 


31 


43 


12 


3 



Summary 



This chapter revealed the results of the accumulated data. Each question was 
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answered with varying degrees of information using different instruments chosen for 
optimal liable measurement. Each question was answered with unbiased data collected by 
the researcher. All 11 participants willfully participated in the research. Some members 
were more descriptive in their journaling than others, which did not alter the findings, but 
rather supported the results of the study. 

Table 8 



Results for Shared Personal Practice 







Pretest 




Posttest 




Item 


4 


3 2 1 


4 


3 2 


1 



1 


1 


3 


5 


2 


2 


6 


3 


0 


2 


0 


3 


3 


5 


2 


4 


3 


2 


3 


0 


2 


3 


6 


0 


2 


4 


5 


4 


0 


5 


2 


4 


4 


5 


2 


1 


5 


0 


3 


4 


4 


1 


6 


4 


0 


6 


0 


6 


4 


1 


0 


9 


2 


0 


Total 


1 


22 


21 


22 


9 


32 


18 


8 



Table 9 

Results for Supportive Conditions for Relationships 


Pretest 




Posttest 




Item 4 3 2 


1 4 


3 2 


1 



1 


5 


4 


2 


0 


4 


5 


2 


0 


2 


0 


8 


3 


0 


0 


10 


1 


0 


3 


0 


8 


1 


2 


4 


6 


1 


0 


4 


2 


6 


1 


2 


3 


8 


1 


0 


Total 


7 


26 


7 


4 


11 


29 


5 


0 
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Table 10 



Results for Supportive Conditions for Structures 







Pretest 




Posttest 




Item 


4 


3 2 1 


4 


3 2 


1 



1 


0 


2 


5 


4 


0 


7 


2 


2 


2 


4 


7 


1 


0 


6 


5 


0 


0 


3 


6 


5 


0 


0 


5 


5 


1 


0 


4 


0 


5 


3 


3 


5 


5 


1 


0 


5 


0 


3 


4 


4 


3 


5 


2 


1 


6 


1 


10 


0 


0 


1 


10 


0 


0 


7 


0 


3 


4 


4 


0 


5 


4 


2 


8 


0 


4 


7 


0 


0 


4 


7 


0 


9 


0 


3 


4 


4 


1 


7 


1 


2 


Total 


11 


43 


28 


19 


21 


53 


18 


7 



The rubric that was used to assess the growth of the leadership team allowed for a 
visual representation of the improved and stagnate behaviors demonstrated by the 
leadership team. Pretest and posttest surveys, while minimal, gave statistical factors of 
achievement that supported the use of a protocol in implementing and effective 
leadership team. The next chapter further discusses the research findings because it goes 
beyond the present data and into a discussion of the interpretation of the data. 
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Chapter 5: Discussion 

Introduction 

The final chapter of the applied dissertation is divided into five sections. Each 
section reviews essential elements of the completed study. The first section is an 
overview of the study. In the opening paragraphs, the researcher summarizes the purpose 
and procedures of the study. The next section shares a discussion of the results from each 
research question. The data are presented in chapter 4; therefore, in chapter 5, the 
researcher elaborates and interprets the data related to each question. Following the 
interpretations are the conclusions found by the researcher with reference to the research 
shared in chapter 2 of the dissertation. Implications of the study precede the section on 
limitations of the study. In the final section, the researcher makes recommendations for 
future research on implementing a protocol to organize and enable effective leadership 
meetings. 

Overview of Applied Dissertation 

This study was conducted on a leadership team in a public middle school. The 
problem on which this study focused was the need to improve the organization and 
effectiveness of leadership meetings. This study took place over a period of 7 months. 
The first 2 months focused on professional-development sessions that included a 
simulation with the leadership team. The subsequent 5 months involved the 
implementation of the protocol and evaluating the protocol use as to the impact on 
organizing and implementing effective leadership meetings. 

In this study, the researcher implemented procedures for evaluating the use of a 
protocol in organizing and implementing effective leadership meetings. A single-group 
research design utilized both quantitative and qualitative measures, making this a mixed- 
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methods evaluation. The Kirkpatrick (1996) evaluation model allowed for evaluation of 
the participants’ reactions to the professional development, skills and knowledge learned 
that cause changes in behavior, and the overall results of the professional development. 

The leadership team was instructed on the purpose, benefits, and use of the 
CCCTAR protocol (RESA, 2002). The leadership team practiced the protocol through a 
simulation. Through observations and use of the rubric for Elements of an Effective 
Collaborative Group (Danielson, 2002), the researcher evaluated team members’ 
professional-collaboration behaviors (e.g., open-mindedness, constructive feedback, 
support of colleagues’ thoughts and opinions, participation, and active listening). The 
researcher analyzed participants’ skills and knowledge based on the data collected using 
the rubric. Based on the data collected, the researcher measured the leadership team 
members’ ability to organize and implement effective leadership meetings. 

Elaboration and Interpretation of Results 

The Kirkpatrick (1996) model of evaluation, around which this study was framed, 
allowed for evaluation of the participants’ reactions to the professional development, 
skills and knowledge learned that cause changes in behavior, and the overall results of the 
professional development. The Kirkpatrick model uses four levels of evaluation to 
evaluate the effectiveness of training: reaction, learning, behavior, and results. The 
research questions sought to address each of the four levels. Below are the research 
questions with an interpretation of the results by the researcher of this study. Included 
with the interpretations of the data is relevant research as it pertains to this study. 
Research Question 1 

What were the leadership team members’ reactions to the protocol as a 
professional-development tool? This was measured using the PDES (Steele, 2007), which 
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was completed after the second training session. Research Question 1 addressed 
Kirkpatrick’s (1996) Level 1, Reaction. 

In Level 1, Kirkpatrick (as cited in Naugle, 2000) described reaction as “how 
participants feel about a variety of segments in the training, their perceptions or feelings 
(positive or negative) about the material and the overall experience” (p. 4). By evaluating 
this initial step, the researcher knew she could move forward with the study. According to 
Naugle, “Kirkpatrick noted that at this level of evaluation, you are not attempting to 
measure any degree of learning, and the usual manner of assessment is a self report from 
participants” (p. 6). 

The PDES (Steele, 2007) reported shared positive responses in regard to the 
professional development. The journal entries shared by the leadership team members 
stated an appreciation for the training in the collaborative process and use of the protocol. 
The reaction to the professional development and the protocol was overwhelmingly 
positive. Research shows that professional development “should be a collaborative 
endeavor, with teachers and administrators working the plan and seeing its 
implementation through to the final stages” (Steele, 2007, p. 14). With the state 
department of education mandating the use of a protocol, the leadership team was 
interested in the purpose and use of a protocol. This interest aided the researcher’s 
objective to establish effective meetings. Meaningful professional development is 
centered on achieving learning goals and supporting learners’ needs. The two staff 
developments in this research study allowed for this process to take place. 

Having ample time to elaborate on the protocol’s purpose, as well as to review the 
necessary skills to collaborate in a professional setting, was ideal in establishing this 
study. The leadership team welcomed the opportunity to try a different method in 
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conducting their leadership meetings. 

Research Question 2 

What skills and knowledge related to organizing and implementing effective 
leadership team meetings will the leadership team members learn as a result of using the 
protocol as a professional development tool? Observational data of the leadership team 
meetings recorded on the rubric for Elements of an Effective Collaborative Group 
(Danielson, 2002) and comments in reflective journals were used to address this question. 
Research Question 2 addressed Kirkpatrick’s (1996) Level 2, Learning. 

According to Kirkpatrick’s (1996) Level 2, individuals need to engage in staff 
development to perfect the skills necessary to improve upon a desired aptitude. In this 
study, the researcher sought to enable implementation of effective and organized 
leadership team meetings with the use of a protocol. After two staff-development 
sessions, one on collaborative behaviors and the other on the use of a protocol, leadership 
team members were given the opportunity to demonstrate their learning in actual 
leadership team meetings using the CCCTAR protocol (RES A, 2002). 

In Level 2, participants are assessed on the skills or knowledge they have gained 
from participating in staff-development sessions. Through interviews and observations, 
as well as reflective journals, a researcher can judge to what degree participants have 
improved upon a skill or knowledge of the given subject (Naugle, 2000). 

The leadership team observed in this study progressively improved upon desired 
skills that would allow for effective and organized meetings. They showed growth in the 
areas of relationships with colleagues, shared and supportive leadership, and 
communication skills, as measured by the rubric for Elements of an Effective 
Collaborative Group (Danielson, 2002). The area that was not influenced by the staff- 
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development training sessions or the use of a protocol was member participation. 
Regardless of the emphasis placed on the importance of all members participating, some 
members felt intimidated due to their lack of knowledge in certain topics of conversation, 
whereas others had few observable attempts or interest in participating in the 
collaborative efforts of the leadership team. However, because the team members 
typically only saw each other at the monthly leadership team meeting, the lack of social 
interaction outside scheduled meetings might have been a contributor to the lack of 
mastery in member participation and establishing relationships with colleagues. 

This conclusion is supported with entries shared by the leadership team members 
in their journals. Some members’ journal entries indicated that they were not familiar 
with other members of the team, and this prevented a certain level of comfort when 
sharing their opinions. Because of the inconsistency of behavior (e.g., members not 
actively participating) and the lack of effort from other members, the leadership team was 
unable to achieve the proficient level in the area of member participation and 
relationships with colleagues, according to the rubric for Elements of an Effective 
Collaborative Group (Danielson, 2002). 

Fullan (1993) stressed that the ability to collaborate is a core requisite of the post- 
modern society. He wrote further about the importance of collaborative skills and the 
ability to form and maintain relationships. He shared that it is not possible to be an agent 
for social improvement without the ability to collaborate professionally (Fullan, 1993). It 
is unfortunate that the leadership team in this study was unable to exceed the proficient 
level in regard to relationships with colleagues. Regardless of the members’ insecurities, 
a professional community must be able to collaborate to make a measurable difference in 
their desired goals (Fullan, 2003). DuFour and Eaker (1998) shared the same thoughts 
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about how teachers could increase the effectiveness of the school when they collectively 
identify and work toward a common goal. 

Based on observations, the leadership team members’ behaviors during meetings 
kept them from achieving the proficient level, as scored by the rubric for Elements of an 
Effective Collaborative Group (Danielson, 2002) in the area of leadership team member 
participation. Observations of members in meetings, which included one member 
nodding off during a roundtable leadership team discussion, supported this conclusion of 
the data. For the leadership team to rate at a proficient level, all members must be 
engaged, or an effort has to be made for all members to be engaged in the discussion. 

Inger (1993) wrote about the importance of placing an emphasis on teacher 
collaboration. He stated that even though an emphasis had been placed on collaboration, 
the success of collaborative efforts had been rare. According to Inger, “there are many 
barriers to teacher collaboration, and the barriers are of many kinds” (p. 10). Leonard and 
Leonard (2003) stated, “Inhibitors to such collegial professional interaction have been 
noted often in the literature, among them time constraints, fragmented visions, 
competitiveness, conflict avoidance, and lack of administrative support” (p. 7). This past 
research is applicable to the current study. Although many of these inhibitors were 
intentionally addressed in the staff developments (e.g., fragmented vision, 
competitiveness, and conflict avoidance), and the protocol (e.g., time constraints), which 
was designed to motivate them, the group was unable to overcome similar barriers. 

One member was angered because she made every effort to get to each monthly 
meeting and voiced her perception that others seemed to think it was an option. 
Attendance was an issue for only one member, who served as a parent liaison for the 
school. Another member often arrived after the meeting had started due to afternoon 
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duties associated with his position as assistant principal. One member observed and 
commented that the state review committee would never deem the school’s leadership 
team as operational, according to the GAPSS process, unless a mandate was made by the 
principal that these meeting were mandatory and that every member should serve for a 
minimum of 2 years. 

Before initiating any change in a team, Pearce and Herbick (as cited in Folkman, 
2003) pointed out that it was important to “manage perceptions that employees have 
regarding the organization and how it supports organizational teams” (p. 38). Although 
the leadership team is the heart of the organization, the principal of the school must first 
set high expectations and place significant value upon those who serve in the leadership 
team (Folkman, 2003). 

Journal responses throughout the study alluded to the displeasure of some 
members for other members’ lack of willingness to take the role of leader. This was 
another area in which the leadership team did not achieve the proficient level. A 
comment from one member’s journal entry expressed frustrations with other members. 
She wrote, “The group was called a team of leaders; yet there seemed to be a lot of 
followers.” Team members were supposed to be a team; however, some members did all 
of the talking, and others just nodded in agreement to whatever was said. Few members 
stepped forward to take the role of the leader or offer opinions that were oppositional to 
the principal’s opinion. Even with the lack of mastery in several areas, the outcomes from 
the leadership team meeting observations indicated a progressive growth in the team’s 
ability to organize and participate in effective leadership meetings. 

The research found this growth of the leadership team to be a direct result of the 
protocol used in this study. McDonald et al. (2007) began creating and supporting the use 
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of protocols in education in 1991. Their first basic idea behind the use of protocols was 
that educators could take charge of their own learning. They stated, “One of the values of 
using protocols as a learning format, in our view, is that they can accelerate the 
development of facilitative leadership and thus assist in the creation of new workplaces 
for educators” (p. 13). This study and the use of the CCCTAR protocol represented the 
necessary first step in implementing an organizational structure to allow the leadership 
team to have effective leadership meetings. 

Research Question 3 

Will leadership team members routinely and consistently apply the new skills and 
knowledge related to organizing and implementing effective leadership team meetings in 
their meetings? This was measured by observing the leadership team meetings and the 
responses shared in the reflective journals. Research Question 3 addressed Kirkpatrick’s 
(1996) Level 3, Behavior. Kirkpatrick (as cited in Naugle, 2000) indicated that the use of 
Level 3 evaluation is to “measure the extent to which participants change their on-the-job 
behavior” (p. 11). This is called transfer training, which involves asking if what was 
learned is being used. The researcher was measuring if participants were using the 
concepts and skills from the staff-development sessions and if they were able to apply 
them in their own behavior. 

Bailey et al. (2004) interpreted the collaborative process as one that expands the 
definition of a leadership role. According to Darling-Hammond (1996), “shared decision 
making is a key factor in reforming curricula and transforming the work of teachers (p. 

7). When schools are structured to facilitate collaboration and expand leadership roles, 
efficacy and the ability to meet students’ needs increase (Rosenholtz, 1998). Although 
many schools have made collaborative efforts through professional-learning 
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communities, Bailey et al. (2004) found that missing from these efforts were purpose, 
connection, and distributed leadership. Without these areas, collaboration can have little 
effect on student leaning. It was the intent of the researcher to make the leadership team 
fully aware of the necessities of their ability to collaborate and make shared decisions. 

Bailey et al. (2004) shared that sustainable school change was possible if the 
principal leader of the school was aware of the necessity of shared leadership (i.e., 
integrated leadership). They stated the importance of knowledge in the areas of research- 
based practices and change management. The design of specific strategies and tools that 
would help the leader better understand the application of leadership meant that 
sustainable school change was possible. 

The principal of the leadership team evaluated in this study was fully aware of the 
need for change. She completely agreed with the use of the protocol and her role 
regarding its effect on the overall process of collaboration and shared decision making. 
She made herself and members of the leadership team aware of current research and 
proven practices in improving collaboration. This study was the means toward improving 
overall student achievement and sustainable change for the target school. 

Table 3 shows the results of the tallied frequency levels for individual behaviors 
(as shown in Table 4) and the calculated averages to determine the leadership team’s 
demonstrated collaborative behaviors as a whole. In Table 3, frequency levels were 
scored using the rubric for Elements of an Effective Collaborative Group. In place of 
averages, the demonstrated behaviors were scored according to levels of proficiency. A 
coefficient value of .99 was deemed a whole number due to decimal placement. Table 3 
gives evidence in the accomplishments of each collaborative behavior as to answer 
Research Question 3, which asked, “Will leadership team members routinely and 
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consistently apply the new skills and knowledge related to organizing and implementing 
effective leadership team meetings in their meetings?” 

As stated previously, Kirkpatrick’s (1996) third level of evaluation is behavior. 
The use of Level 3 evaluations is to “measure the extent to which participants change 
their on- the -job behavior” (Naugle, 2000, p. 1 1). The researcher looked at the relevance 
of the change and the sustainability of such change, as illustrated in Tables 3 and 4. 
Research Question 4 

Research Question 4 asked, “What result will learning the protocol have on 
participants’ judgments about organizing and implementing effective leadership 
meetings?” The pretest and posttest surveys that addressed members’ perceptions of roles 
and professional-collaboration abilities experienced during meetings were used to 
measure the effectiveness of the protocol. This question addressed Kirkpatrick’s (1996) 
Level 4, Results. 

Level 4 of Kirkpatrick’s (1996) training evaluation model measures results of the 
training. Naugle (2000) summarized this stage as the formation of a basis of learning 
upon which to build, the development of skills applied to participants’ learning, and the 
life acquisition of skills to make personal improvements. Measurements of the program’s 
success at Level 4 were based on posttest surveys, observations, and journal reflections 
(Naugle, 2000). 

The null hypothesis was that the use of the protocol would have no effect on the 
participants of the study. The researcher set the level of significance at .05. The number 
of participants for each question was 11. 

An analysis of the overall project showed that z = -3.929, which rejected the null 
hypothesis. This statistically shows that, overall, the protocol had a statistically 
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significant effect on participants’ judgments about organizing and implementing effective 
leadership meetings. However, this can be broken down into the six subsections, which 
show mixed results. 

The researcher found that in the following subsections, a statistically significant 
difference was measured in participants’ judgments: (a) shared and supportive leadership 
(z = -2.535), (b) shared value and vision (z = -2.111), (d) shared personal practice (z = 
-2.082), and (f) supportive conditions for structures ( z = -1.685). 

The following subsections did not show a statistically significant difference 
between pretest and posttest survey scores in participants’ judgment before and after the 
professional development and use of the protocol: (c) collective learning and application 
(z. = -1.054) and (e) supportive conditions for relationships (z = -1.315). It is unclear if 
these areas contributed to any change in participants’ judgments about organizing and 
implementing effective leadership meetings. 

Results may have been more conclusive with a larger data set. In any case, the 
researcher could conclude that following the protocol made a difference in improving 
participants’ judgments about organizing and implementing effective leadership 
meetings. 

Limitations 

This study was limited to the implementation of professional development and 
evaluation of the effects of such on one school leadership team. Possible threats to the 
internal validity of the study were the participants’ unwillingness to be completely honest 
in their journal and survey responses. Although complete confidentiality was 
contractually agreed upon, underlining fears may have been present on behalf of the 



participants. 
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Another limitation was that of the small sample size. Although the study would 
remain valid, the size of the group lessened the chances of varied responses, hence 
minimizing the generalizability of the study’s findings. In addition to the small sample 
size, the professional development and implementation of the protocol took place at only 
one site. These conditions limit the generalizeability of this study. 

Recommendations 

This section is divided into two parts. The first part discusses recommendations 
for practice, and the other section contains recommendations for future research. 

Recommendations for practice. To better serve the leadership team and other 
professional-learning communities at the school, the researcher suggests continued use of 
the protocol. Once faculty members become familiar with the steps and the importance of 
the protocol in organizing effective meetings, they may move forward in their abilities to 
collaborate effectively as well. With effective collaboration taking place, the impact on 
student learning is wide ranging. 

The researcher of this study recommends that the leadership team continue to use 
the protocol during leadership meetings. Although the leadership team had not yet 
mastered all the collaborative skills addressed in the study, they were well on their way in 
applying the principals of collaboration. The CCCTAR protocol allowed the leadership 
team to experience organized leadership meetings. Five organized meetings made it 
possible for effective communication and the beginning of developing effectual 
collaboration. This study was an introduction to what could become an optimal 
experience for all members of the leadership team. Practice and continued fostering of 
professional relationships would allow for the growth of this leadership team. 

Other professional learning communities within the school would benefit from the 
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use of the protocol as well. Using a protocol provides a sense of professionalism and 
develops relationships that communicate in a professional manner rather than personal 
approach. As the data showed, using the protocol was not the cure-all to effective 
collaboration, but it was a step in the right direction. 

Recommendations for future research. More research is needed in the area of 
school leadership teams. Much of what was available at the time of this dissertation was 
limited to industrial leadership teams (i.e., groups). The research that was available for 
school leadership teams was inconclusive and vague. 

Although articles were numerous about the need to collaborate, much of the 
information shared referred to individuals functioning in a group. There is a need for 
research about teams (i.e., groups) and their ability to collaborate and the impact of that 
collaboration on student achievement. This study represented only a beginning in a much 
needed research endeavor. 

Future researchers on this topic are advised to host more staff-development 
sessions on the importance of collaborating. It would be effective to show videos where 
the viewing of effective collaboration is taking place. To be introduced to the theory 
behind collaboration is merely not enough to have a significant impact on research 
participants. The practice of journaling is recommended to continue. The journal entries 
provide the researcher an insight into to the individual members’ thoughts and 



understanding of collaborating. 
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Leadership Areas 

a) Governance - The organizational structure, learning environment, and academic achievement 
of all learners function as the central focus of school planning and management. A consistent and 
sustained commitment to engaging the input and involvement of representative stakeholders 
among student, family, and community groups characterizes a key element of effective school 
governance. Seven key elements frequently comprise the focus of school governance: policies 
and practices, leadership, teacher qualifications, system support, decision making/problem 
solving, allocation of resources, and facilitating the change process. 

b) Problem solving - When barriers and obstacles impede the academic achievement or 
organizational productivity within a school site, collaborative teams engage in a puiposeful and 
structured approach to collaborative problem-solving, generally involving the following steps: (1) 
identifying the problem; (2) framing the problem as a statement or research questions;(3) 
collecting and analyzing data and presenting the data to staff and other stakeholders, delineating 
potential sources and causes at the root(s) of the identified problem(s); (4) using insights and 
conclusions from this data to generate potential solutions to the articulated problem(s); (5) 
building consensus about appropriate final problem solutions, (6) generating a viable action plan 
to address the problem; and (7) monitoring and evaluating the effectiveness and impact of the 
problem based upon sustained data analysis and presentation. 

c) Decision-making - As representative stakeholder groups meet to determine appropriate 
programs and processes as well as to analyze the causes of academic and organizational problems 
and related solutions, they use a cohesive and sustained process to arrive at consensus-driven 
decisions. Generally, this process involves the following stages: (1) Identify the reasons and 
generate a rationale for the proposed decision(s) being investigated; (2) delineate the range of the 
decisions to be discussed, including initial discussion of the importance and timeliness of the 
decision(s) being investigated; (3) frame the decision in consensus-based language; (4) engage in 
scenarios and projection discussions of the potential impact and effects of various decision 
outcomes; (5) determine an action plan, including monitoring strategies; (6) implement the 
decision(s) based upon the final consensus-driven timeline and action plan; (7) monitor the 
ongoing impact and value added of the decision(s); and (8) integrate this decision-making process 
into the school improvement planning process, with continuing modification of decisions being 
implemented. 

d) Distributed leadership - The principal guides the process of decision making and problem 
solving in such a way that all staff members have opportunities to provide input and to assume 
leadership positions, where appropriate. 

e) Experiences created - The school organizational structure is designed to create and sustain 
experiences for teachers to serve as instructional leaders within the school. 

f) Instructional leadership development - Teacher leaders participate in instructional leadership 
development experiences and serve in a variety of instructional leadership roles. 

g) Team approach - In order to build a cohesive team, the principal and school leadership team 
subscribe to collaborative efforts for decision making. 
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Professional Learning Community Assessment 



Project Overview 


Pretest 

Results 


Posttest 

Results 


4 = Strongly Agree and 
1 = Strongly Disagree 


D 


3 


2 


■ 


fl 


3 


2 


■ 


Shared and supportive leadership 




1. The staff is consistently involved in discussions 
and making decisions about school issues 


0 


4 


3 


4 


2 


5 


2 


2 


2. The principal incorporates advice from staff to 
make decisions 


0 


4 


6 


1 


3 


6 


2 


0 


3. The staff has access to key information 


1 


3 


5 


2 


4 


4 


2 


1 


4. The principal is proactive and addresses area 
where support is needed 


5 


4 


2 


1 


6 






0 


5. Opportunities are provided for staff to initiate 
change 


0 


3 


2 


6 


0 








6. The principal participates democratically with 
sharing power and authority 


0 


3 


4 


4 


1 


5 


5 




7. The principal shares responsibility and rewards 
innovative actions 


1 


8 


2 


0 




8 


1 


0 


8. Leadership is promoted and nurtured among staff 


0 


5 


6 


0 




7 


3 


0 


9. Decision making takes place through 
communication across grade and subject areas 


0 


2 


3 


6 








0 


10. Stakeholders assume shared responsibility and 
accountability for student learning without evidence 
of imposed power of authority 


1 


1 


5 


4 


1 


2 


5 


3 


Shared and supportive leadership tot: 


8 


37 


38 


28 


23 


52 


29 


8 


Shared value and vision 




1. A collaborative process exists for developing a 
shared sense of values among staff 


0 


7 


3 


1 


1 


8 


2 


0 


2. Shared values support norms of behavior that 
guides decisions about teaching and learning 


2 


6 


1 


2 


3 


7 


0 


1 


3. The staff shares vision for school improvements 
that have an undeviating focus on student learning 


0 


3 


4 


4 


3 


4 


4 


0 


4. Decisions are made in alignment with the 
school's values and vision 


2 


7 


2 


0 


4 


7 


0 


0 


5. A collaborative process exists for developing a 
shared vision among staff 


0 


3 


3 


5 


5 


4 


2 


0 


6. School goals focus on student learning beyond 
test scores and grades 


3 


5 


3 


0 


1 


7 


3 


0 
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7. Policies and programs are aligned to the school's 
vision 


3 


8 


0 


0 


3 


8 


0 


0 


8. Stakeholders are actively involved in creating 
high expectations that serve to increase student 
achievement 


0 


6 


3 


2 


3 


6 


2 


0 


Shared valued and vision tot: 


10 


45 


19 


14 


23 


51 


13 


1 


Collective learning and application 




1. The staff works together to seek knowledge, 
skills, and strategies and apply this new learning to 
their work 


2 


8 


1 


0 


3 


7 


1 


0 


2. Collegial relationships exist among staff that 
reflect commitment to school improvement efforts 


3 


6 


2 


0 


6 


5 


1 


0 


3. The staff plans and works together to search for 
solutions to address diverse student needs 


0 


4 


2 


5 


5 


4 


2 


0 


4. A variety of opportunities and structures exist for 
collective learning through open dialogue 


0 


3 


4 


4 


0 


6 


2 


3 


5. The staff engages in dialogue that reflects a 
respect for diverse ideas that lead to continued 
inquiry 


1 


7 


3 


1 


2 


6 


3 


0 


6. Professional development is focused on teaching 
and learning 


6 


5 


0 


0 


7 


4 


0 


0 


7. School staff and stakeholders learn together and 
apply new knowledge to solve problems 


2 


4 


3 


2 


2 


6 


3 


0 


8. The school staff is committed to programs that 
enhance learning 


7 


4 


0 


0 


6 


5 


0 


0 


Collective learning and application tot: 


21 


41 


15 


12 


31 


43 


12 


3 


Shared personal practice 




1. Opportunities exist for staff to observe peers and 
offer encouragement 


1 


3 


5 


2 


2 


6 


3 


0 


2. The staff provides feedback to peers related to 
instructional practices 


0 


3 


3 


5 


2 


4 


3 


2 


3. The staff informally shares ideas and suggestions 
for improving student learning 


0 


2 


3 


6 


0 


2 


4 


5 


4. The staff collaboratively shares ideas and 
suggestions for improving student learning 


0 


5 


2 


4 


4 


5 


2 


1 


5. Opportunities exist for coaching and mentoring 


0 


3 


4 


4 


1 


6 


4 


0 


6. Individuals and teams have the opportunity to 
apply learning and share the results of their practice 


0 


6 


4 


1 


0 


9 


2 


0 


Shared personal practice tot: 


1 


22 


21 


22 


9 


32 


18 


8 
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Supportive conditions: relationships 


1. Caring relationships exist among staff and 
students that are built on trust and respects 


5 


4 


2 


0 


4 


5 


2 


0 


2. Outstanding achievement is recognized and 
celebrated regularly in our school 


0 


8 


3 


0 


0 


10 


1 


0 


3. A culture of trust and respect exists for taking 
risks 


0 


8 


1 


2 


4 


6 


1 


0 


4. School staff and stakeholders exhibit a sustained 
and unified effort to embed into the culture of the 
school 


2 


6 


1 


2 


3 


8 


1 


0 


Supportive condition relationships tot: 


7 


26 


7 


4 


11 


29 


5 


0 


Supportive conditions: structures 


1. Time is provided to facilitate collaborative work 


0 


2 


5 


4 


0 


7 


2 


2 


2. The school schedule promotes collective learning 
and shared practice 


4 


7 


1 


0 


6 


5 


0 


0 


3. Fiscal resources are available for professional 
development 


6 


5 


0 


0 


5 


5 


1 


0 


4. Appropriate technology and instructional 
materials are available to staff 


0 


5 


3 


3 


5 


5 


1 


0 


5. Resource people provide expertise and support 
for continuous learning 


0 


3 


4 


4 


3 


5 


2 


1 


6. The school facility is clean, attractive, and 
inviting 


1 


10 


0 


0 


1 


10 


0 


0 


7. The proximity of grade level and departmental 
personnel allows for ease in collaborating with 
colleagues 


0 


3 


4 


4 


0 


5 


4 


2 


8. Communication systems promote a flow of 
information among staff. 


0 


4 


7 


0 


0 


4 


7 


0 


9. Communication systems promote a flow of 
information across the entire school community, 
including central office personnel, parents, and 
community members 


0 


3 


4 


4 


1 


7 


1 


2 


Supportive Conditions Structures Tot: 


11 


42 


28 


19 


21 


53 


18 


7 


OVERALL: 


58 


213 


128 


99 


118 


260 


95 


27 
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Professional Development Evaluation Survey 

Topic: Effective Collaboration Through the Use of a Protocol 
Date: 

What concepts or ideas were new to you today? 

1 . 

2 . 

3. 



I would like to know more about the following concepts. 

The staff development objectives were clearly identified and accomplished. 



Strongly agree 


Somewhat 

agree 


Neutral 


Somewhat 

disagree 


Strongly 

disagree 


5 


4 


3 


2 


1 



I have more knowledge and understanding of what a protocol is. 



Strongly agree 


Somewhat 

agree 


Neutral 


Somewhat 

disagree 


Strongly 

disagree 


5 


4 


3 


2 


1 



I learned specific skills needed in effective professional collaboration. 



Strongly agree 


Somewhat 

agree 


Neutral 


Somewhat 

disagree 


Strongly 

disagree 


5 


4 


3 


2 


1 



I believe the use of a protocol will help our leadership team function as operational. 



Strongly agree 


Somewhat 

agree 


Neutral 


Somewhat 

disagree 


Strongly 

disagree 


5 


4 


3 


2 


1 



My reaction or feeling about this staff development is 
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Permission to Use Survey 



To: Erika Anderson 

From: Dr. Julia M. Steele 

Re: Professional Learning Form 



You have my permission to use the Professional Evaluation Form (Professional 
Development Evaluation Survey) used in my dissertation entitled: Implementing and 
Evaluating Teacher Study Groups to Support Change in Teacher Behavior, 2007. 
Please feel free to contact me for more information. 

Contact information: 

130 Oakdale Road 
Griffin, Georgia 30224 
770-227-0461 
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Rubric for Elements of an Effective Collaborative Group 



Elements 


Mastered (4) 


Proficient (3) 


Sufficient (2) 


Needs Work (1) 


Relationships 
With colleagues 


Support and 
cooperation 
characterize 
relationships with 
colleagues. Members 
Take initiative 
in assuming 
Leadership roles. 


Support and 
cooperation 
characterize 
relationships with 
colleagues. 


Members maintains 
cordial 
relationships 
to fulfill 

the duties that the 
school required. 


Member’ s 
relationships with 
colleagues are 
negative and self- 
serving. 


Shared and 
Supportive 
Leadership 


All leadership team 
members actively 
participate and share 
in the decision 
making process 
democratically with 
each other. 


Leadership team 
members participate 
in making group 
decisions with each 
other. 


Leadership team 
members make 
decisions. 


No group 
decisions making 
is taking place. 


Communication 

Skills 


All leadership team 
members were 
communicating 
effectively including 
listening .having an 
open mind, and. 
providing constructive 
feedback. 


Leadership team 
members were 
communicating, and 
some members 
were listening and 
providing feedback. 


Leadership team 
members were 
communicating. 


No positive 
communication 
skills were be- 
ing used. 


Protocol and 
documentation 


All leadership team 
members followed 
protocol as well as 
documentation of 
the meeting. 


Leadership team 
members followed 
protocol. 


Leadership team 
members 
documented 
session. 


No protocol 
and/or 

documentation 
took place. 



Leadership Team 
Members 
Participations 



Leadership team 
members ensure that 
all voices are 
heard in the 
in the discussion. 



Leadership team 
members engage 
all members in the 
in the discussion. 



Leadership team 
members engage 
all members in the 
discussion with 
limited success. 



Only a few 
members 
participate 
in the 
discussion. 



Note. Adapted from Enhancing Student Achievement: A Framework for Student Achievement, by C. 
Danielson, 2002, Alexandria, VA: Association for Supervision and Curriculum Development. 
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Guidelines for Journal Entries 

The following six journal entry segments will be posted for leadership team member to 
address in their reflective journals (small, numbered spiral bond notebooks). The 
segments will be given to leadership team members at the beginning of each monthly 
meeting in the form of a handout. Leadership team members will be given a week’s time 
to complete their journal entries. Along with the specific questions posed, members are 
encouraged to share their own questions, frustrations, and successes they may be feeling 
in regards to improved professional collaborating and shared decision making. Each entry 
will be dated and the role of the person who is writing in the journal will be identifying as 
to either an administrator or teacher. 

Discussion for Journal Entry 1: In your own words, what are some of the appropriate 
behaviors demonstrated by those who collaborate effectively on a professional level? 
What behaviors could you begin to develop that might enable the leadership group to 
function operationally? 

Discussion for Journal Entry 2: Do you feel you and your peers have had enough 
training on what professional collaboration looks and sound like? Do you feel the 
leadership team is functioning operational according to the GAPSS standards? If not, 
where on the GAPSS continuum do you feel the leadership team’s professional 
collaboration would rank? What more do you think should be done in the way of 
professional development to help the team progress? 

Discussion for Journal Entry 3: Where there any significant behavior changes on your 
behalf or those of your peers that have begun to show improved collaboration? If so, what 
specific behaviors? 

Discussion for Journal Entry 4: Does the use of the protocol seem to keep the group on 
focus and allow for optimal professional collaboration? What needs to occur before the 
leadership team can become fully operational in an effective leadership committee? 

Discussion for Journal Entry 5: How have you improved in your ability to professional 
collaborate? Can you identify any changes from last year’s meetings to this year’s 
meetings in the way of shared decision making and professional collaboration? 

Discussion for Journal Entry 6: Have the leadership team members improved in their 
abilities to collaborate effectively with the use of a protocol? Do you feel the leadership 
team is functioning operational according to the GAPSS standards? If not, where on the 
GAPSS continuum do you feel the leadership team’s professional collaboration would 
rank? 
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Agenda for Session 1 



Goals for session: 

1. Discuss and present protocol’s purpose and benefits. Members of the leadership team 

will demonstrate their understanding of the protocol’s use and benefits with their ability 
to transfer their learning in utilizing the protocol during their monthly meetings. 

2. Describe and validate use of reflective journals. Leadership team members will recall 

their learning from the staff development training session as to the use and benefits of 
the reflective journals in their composition of their reflective journal responses 
throughout the study. 

3. Appraise training session. Leadership team members will fill out and submit the 

professional development survey at the conclusion o f the staff development. Members 
will paraphrase the session’s main points and rate their own understanding of key 
concepts shared in the session. 

I. Protocol 

A. Intended use 

1. Through the GAPSS evaluation process, a school leadership team is not 
considered fully operational until, “ The school leadership team has developed 
and consistently uses a protocol for handling business, making decisions, and 
solving problems effectively and collaboratively related to all facets of student 
needs, staff productivity, and organizational performance” (GADOE, 2008). 

B. Purpose 

1. Protocols are steps in talking and listening, describing and judging as 
well as reaching a consensus in decisions. Protocols are prescribed steps to be 
followed forcing transparency in conversations. They segment elements of the 
conversation where it otherwise may blur (McDonald, Mohr, Dichter, & 
McDonald, 2007). 

C. Benefits 

I .Barbara Kohm (2002) wrote in her article Improving Faculty 
Conversation , “...what we don’t know can hurt us” (fl). Kohm revealed how 
important it is to have honest and critical feedback, as it is needed in order to 
effectively communicate within a school faculty. Seeking a positive change in a 
school often lies in the hands of the faculty. 

2. The power of the protocol lies in its reflective practice, genuine listing, 
critical thinking and feedback (Lambert, 2003, p.24). At Sir Winston Churchill 
High School, educators used protocols to examine curriculum leaders’ 
philosophies regarding education. They found that the protocol allowed them to 
hear more voices, more listening occurred and a deeper understanding of the 
issued emerged (Lambert, 2003, p.24). 

3. A protocol helps to develop the attitudes, mindset and skills of teachers 
as they work collaboratively to attain preset goals. McDonald, Mohr, Dichter & 
McDonald (2007) began creating and supporting the use of protocols in education 
in 1991. Their first basic idea behind the use of protocols was that educators can 
take charge of their own learning (p.l). The authors feel that protocols may 
encourage an environment of learning. Protocols force transparences in 
conversations that allow for the participants to gain a deeper understanding of 
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their colleagues’ opinions and insights. “One of the values of using Protocols as a 
learning format, in our view, is that they can accelerate the development of 
facilitative leadership and thus assist in the creation of new workplaces for 
educators” (p.13). 

4. One of the characteristics of protocol is the use of reflective practices. A 
leader of learners is apt to use a protocol to help participants learn to actively 
participate in effective collaboration. “Protocols help imagine alternatives to 
ordinary habits of working together, learning and leading” (McDonald, Mohr, 
Dichter & McDonald, 2007, p.15). 

D. Process of the protocol 

1. The protocol created by the Georgia RES A Network (2002) involves 
setting a designated time and date for groups to meet. At the beginning of each 
session the steps of the protocol are reviewed. Prior to meeting an agenda is sent 
to all participants. This allows time for participants to prepare for the topic of 
discussion set for the meeting. 

II. Reflective journals 

A. Evaluation purpose of study 

B. Use of during evaluation process 

III. Distribute Professional Development Evaluation Survey (Steele, 2007) 
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Appendix H 
Agenda for Session 2 
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Agenda for Session 2 



Goals for session: 

1. Demonstrating and interpret the Steps of Creating a Capacity for Change Through 

Action Research Protocol. Leadership team members will transfer their learning of the 
protocol’s steps in the actual use of the steps in their monthly leadership meetings. 

2. Define and demonstrate effective behaviors of professional collaboration. Leadership 

team members will examine their understanding of effective collaborate behaviors in 
the first composition in their reflective journals. They too will model the learned 
appropriate behaviors in their monthly leadership meeting with their ability to 
professional collaborate effectively. 

I. Creating a Capacity for Change Through Action Research Protocol 

A. Process of the protocol 

1 . The protocol created by the Georgia RES A Network (2002) involves 
setting a designated time and date for groups to meet. At the beginning of each 
session the steps of the protocol are reviewed. Prior to meeting an agenda is sent 
to all participants. This allows time for participants to prepare for the topic of 
discussion set for the meeting. 

B. Steps 

1. Introduction (5 minutes) reviewing the agenda and setting goals for the 
meeting. 

2. Teacher preparation (5-10 minutes) presenter describes the context of 
the meeting and shares any sample work needed to aid in the 
understanding of meeting’s purpose. 

3. Discussion (30 minutes suggested, but can be adjusted to suit needs of 
the group) allows group to actively participate to obtain goals of the 
meeting. 

4. Clarifying Questions (10 minutes, minimum) an activity where 
participants’ reflect on what has been shared. Questions that seek clarity or 
summarize main points of the meeting are presented. The facilitator is the 
one to answer the questions based on their interpretation of key points 
shared by the group. 

5. Debrief (5 minutes) during this time the group discusses any 
fmstrations, misunderstandings, or positive reactions to the session. 

6. Distribute journals 

II. Effective Behaviors of Professional Collaboration 

A. Marzano (2003) establishes three characteristics of importance in 
collaborative behaviors: optimism, honesty, and consideration. 

1. Optimism increases teachers’ self-esteem and motivation. 

2. “Honesty is characterized by truthfulness and consistency between 
words and actions” (p. 177). 

3. Consideration “is sometimes referred to as a people orientation or a 
concern for people” (p. 178). Honesty and consideration both help 
build interpersonal professional relationships. 

Marzano, R. (2003). What works in schools. Alexandria, VA: Association for Supervision and 
Curriculum Development. 
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4. Support of Colleagues- non-judgmental comments and facial reactions 
to comments. No speaking when others are speaking 

5. Active Participation- listening and acknowledging with head nods, 
sharing related comments and ideas at appropriate time (support of 
colleagues), rephrasing a misunderstood comment 

6. Open minded- other’s ideas are valid, be willing to listen to and try 

7. Democratic decision making- majority rules. Excepting you may lie in 
the minority and have trust in your colleagues decisions (Support of 
Colleagues) 

8. Providing constructive feedback- when disagreeing, do not make it 
personal. If the idea is not similar to yours, focus on the idea not the 
person sharing it (Support of Colleagues) 

9. Leadership roles- if you are an expert in an area volunteer to share 
your expertise 

III. Journal Evaluation Entry One 

A. Discussion for journal entry one: In your own words what are some of the 
appropriate behaviors demonstrated by those who collaborate effectively on a 
professional level? What behaviors could you begin to develop that might enable 
the leadership group to functional operationally? 
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Appendix I 
Simulation Exercise 
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Simulation Exercise 

I. In this simulation the teachers are presented with the challenge of 
rectifying a new policy on discipline. Prior to the simulation teachers 
and administrators are asked to think about and come up with suggestion 
on how to improve school wide discipline procedures (for simulation 
purpose only). 

II. The leadership committee will gather for a period of 70 minutes. A timer 
will be kept marking the transitions of the protocol segments. 

III. The goal of the meeting is to come up with three ideas for a new school 
wide discipline policy. 

IV. Rubric: Elements of Effective Collaborative Group (Danielson, 2002) is 
handed out and participants are asked to circle where they think the 
group will score. The lead teacher will move into his presentation of the 
material and follow it with the 30-minute discussion period (agenda 
follows outline). It is in the 30-minute discussion where effective 
collaboration behaviors are expected of the participants. The lead 
teacher will model his own questions and comments with characteristic 
of effective collaboration (open minded, constructive feedback given, 
supportive of colleagues’ thoughts and opinions, participation, and 
active participation). 

V. In the next 10 minutes of the protocol, the lead teacher will review key 
points of the discussion and present the three new ideas for a discipline 
policy. He will be seeking agreement from the group. In this segment 
active listening skills; head-nodding and positive facial reactions are 
expected and sought from the participants. The lead teacher will guide 
the participants in this collaborative behavior buy stating the appropriate 
responses at the appropriate times. The lead teacher will share how the 
non-verbal behaviors signify agreement without interrupting the flow of 
the meeting. 

VI. Additional ten minutes slotted in the simulation for reviewing how the 
group felt they collaborated. Discussions of the rubric scorings will 
occur. 

VII. Practice journaling will be a quick write asking if the group was pleased 
with their efforts in the simulation. 

Simulation Agenda 

* Handout of Rubric; Elements of an Effective Collaborative Group (Danielson, 2002) - 

Leadership Committee members preevaluate where they think the group will perform 
in the simulation activity 

• Introduction (5min) Topic for discussion; Schoolwide discipline policy 

Goal of today’s meeting- Three discipline policies to share with faculty 
(seeking feedback- return in one month with notes from peer’s reaction to the 
suggested new policies/ ideas) 

• Teacher Preparation (5-10min) Review of current discipline policy (lead 
teacher presents information) 

List strengths vs. weaknesses (what has worked vs. what has not- 
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statistical representation of write up referrals vs. changed behaviors) 

• Open discussion (30min) (round table- leadership committee members) 
Suggestion for modification to eliminate weakness of current plan 

New ideas 

• Clarifying Questions and Answers (lOmin) (lead teacher summarizes main point 
and restates new ideas shared by leadership committee members) 

Three new policies/ideas agreed upon for faculty review 

• Reaction to Meeting from participants (lOmin) (leadership committee 
members) 

Re-evaluate group’s actual performance on the Rubric; Elements of an 
Effective Collaborative Group (Danielson, 2002) - Lead teacher will host the 
review and discussion of the rubric scores 
*Frustrations/ misconception 
*Praise 

• Journaling (for simulation they will be quick writes due at the conclusion 
of meeting)(leadership committee members) 
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Protocol for Professional Development 
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Protocol for Professional Development 

1. Introduction (5 minutes) reviewing the agenda and setting goals for the 
meeting. 

2. Teacher preparation (5-10 minutes) presenter describes the context of the 
meeting and shares any sample work needed to aid in the understanding of meeting’s 
purpose. 

3. Discussion (30 minutes suggested, but can be adjusted to suit needs of the 
group) allows group to actively participate to obtain goals of the meeting. 

4. Clarifying Questions (10 minutes, minimum) an activity where participants’ 
reflect on what has been shared. Questions that seek clarity or summarize main points of 
the meeting are presented. The facilitator is the one to answer the questions based on their 
interpretation of key points shared by the group. 

5. Debrief (5 minutes) during this time the group discusses any frustrations, 
misunderstandings, or positive reactions to the session. 



6. Distribute journals. 
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Appendix K 

Effectiveness of Professional-Development Workshop 
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Effectiveness of Professional-Development Workshop 



Professional Development Workshop Effectiveness 




■ Strongly Agree □ Somewhat Agree □ Neutral □ Somewhat Disagree □ Strongly Disagree 
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Appendix L 

Summary of Calculations for Pretest and Posttest Surveys 
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Summary of Calculations for Pretest and Posttest Surveys 
OVERALL SAMPLE: 

Two-sample Wilcoxon rank-sum (Mann- Whitney) test 

post I obs rank sum expected 

+ 

01 45 1561.5 2047.5 

1 I 45 2533.5 2047.5 

+ 

combined I 90 4095 4095 

unadjusted variance 15356.25 
adjustment for ties -53.60 



adjusted variance 15302.65 

Ho: score(post==0) = score(post==l) 
z = -3.929 
Prob > Izl = 0.0001 



SECTION 1 Shared and supportive leadership RESULT: 
Two-sample Wilcoxon rank-sum (Mann- Whitney) test 

post I obs rank sum expected 

-+ 

01 10 71.5 105 

1 I 10 138.5 105 

+ 

combined I 20 210 210 

unadjusted variance 175.00 
adjustment for ties -0.39 



adjusted variance 174.61 

Ho: score(post==0) = score(post==l) 
z = -2.535 
Prob > Izl = 0.0112 
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SECTION 2 Shared value and vision RESULT: 
Two-sample Wilcoxon rank-sum (Mann- Whitney) test 

post I obs rank sum expected 

---+ 

018 48 68 

118 88 68 

+ 

combined I 16 136 136 

unadjusted variance 90.67 

adjustment for ties -0.93 



adjusted variance 89.73 

Ho: score(post==0) = score(post==l) 
z = -2.111 
Prob > Izl = 0.0347 



SECTION 3 Collective learning and application RESULT: 
Two-sample Wilcoxon rank-sum (Mann- Whitney) test 

post I obs rank sum expected 

---+ - 

018 58 68 

118 78 68 

+ 

combined I 16 136 136 

unadjusted variance 90.67 

adjustment for ties -0.67 



adjusted variance 90.00 

Ho: score(post==0) = score(post==l) 
z = -1.054 
Prob > Izl = 0.2918 
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SECTION 4 Shared personal practice RESULT: 
Two-sample Wilcoxon rank-sum (Mann- Whitney) test 

post I obs rank sum expected 

---+ 

016 26 39 

116 52 39 

+ 

combined I 12 78 78 

unadjusted variance 39.00 
adjustment for ties 0.00 



adjusted variance 39.00 

Ho: score(post==0) = score(post==l) 
z = -2.082 
Prob > Izl = 0.0374 



SECTION 5 Supportive conditions: relationships RESULT: 
Two-sample Wilcoxon rank-sum (Mann- Whitney) test 

post I obs rank sum expected 

+ 

01 4 13.5 18 

1 I 4 22.5 18 

-+ 

combined I 8 36 36 

unadjusted variance 12.00 

adjustment for ties -0.29 



adjusted variance 1 1 .7 1 

Ho: score(post==0) = score(post==l) 
z = -1.315 
Prob > Izl = 0.1886 
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SECTION 6 Supportive conditions: Structures RESULT: 
Two-sample Wilcoxon rank-sum (Mann- Whitney) test 

post I obs rank sum expected 

- ---+ 

01 9 66.5 85.5 

1 I 9 104.5 85.5 

+ 

combined I 18 171 171 

unadjusted variance 128.25 
adjustment for ties - 1 .06 



adjusted variance 127.19 

Ho: score(post==0) = score(post==l) 
z = -1.685 
Prob > Izl = 0.0920 




